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ABSTRACT 

A 2~year project was undertaken to develop 
performance standards for vocational and applied technology education 
in Texas. During the first project year, background research on 
performance standards in vocational and applied technology education 
was conducted and model performance standards for selected vocational 
programs were delineated. During the second project year, the process 
of developing performance standards for sequences of vocational 
courses preparing students for 22 priority and/or emerging 
occupations was continued. In addition, course-specific core 
performance standards were developed for those vocational courses in 
which they were needed, and a feasibility study of the new 
performance standards was conducted in a sample of 12 Texas school 
districts. A data-gathering instrument to be used by local vocational 
administrators in collecting data for their required district 
evaluations was developed. (Appendixes include an abstract of the 
1991-1992 project work plan, advisory committee minutes, competency 
lists, task list summaries, a prototype data collection instrument, 
an occupations worksheet, tables matching existing Student 
Occupational Competency Achievement Tests to essential elements for 
specific courses, and project progress reports. Contains 26 
references and 14 tables.) (MN) 
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Section 1. PROBLEM AREA 



Anticipating that the Carl D. Perkins Act of 1990 would include new and more 
demanding requirements for evaluating vocational and applied technology education 
programs, the Texas Education Agency (TEA) granted funds to the Extension Instruction 
and Materials Center (EIMC) at The University of Texas at Austin to conduct a two- 
year project to research and develop measurable and objective criteria to assess the 
quality and performance of vocational and applied technology education in secondary 
schools in Texas. The project Performance Standards for Vocational and j^plied 
Technology Education was initiated July 1, 1990. The Carl D. Perkins Vocational and 
Applied Technology Education Act Amendments of 1990, enacted in September 1990, 
authorizes Federal assistance for vocational education through Fiscal Year 1995. The 
Act makes several significant amendments to focus the Federal funds on improving 
vocational education and, in particular, on improving education and services for 
members of special populations, including disabled and disadvantaged individuals.-^ 



Requirements of the Carl D. Perkins Act of 1990 

Section 115 of the Perkins Act of 
1990 is entitled "State and local 
Standards and Measures." It requires 
development and implementation of a 
statewide system of core standards and 
measures of performance before July 
1992. The system must include (1) 
measures of learning and competency 
gains, including student progress in the 
achievement of basic and more advanced 
academic skills; (2) one or more measures 
of performance, which shall include only- 
(A) competency attainment; (B) job or 
work skill attainment or enhancement including student progress in achieving 
occupational skills necessary to obtain employment in the field for which the student has 
been prepared; (C) retention in school or completion of secondary school or its 
equivalent; and (D) placement into additional training or education, military service, or 
employment; (3) incentives or adjustments that are-(A) designed to encourage service to 
targeted groups or special populations; and (B) for each student, consistent with the 
student's individualized education program developed under section 614(a)(5) of the 
Education of the Handicapped Act, where appropriate; and (4) procedures for using 
existing resources and methods developed in other programs receiving Federal assistance. 

Development of the statewide system is to take into consideration standards and 
measures develop)ed under job opportunities and basic skills training programs that meet 



Each State board receiving funds under 
this Act shall develop and implement a 
statewide system of core standards and 
measures of performance for secondary 
and postsecondary vocational education 
programs. 

Perkins Act cf 1990, Sec. 115 



the requirements of section 402(a)(19) of the Social Security Act and standards described 
under Section 106 of the Job Training Partnership Act (JTPA). For evaluating the 
performance of youth programs, these standards and measures include (A) placement in 
unsubsidized employment; (B) retention in employment; (C) increase in earnings; (D) 
reduction in number of individuals and families receiving cash welfare payments; (E) 
attainment of recognized employment competencies; (F) elementary, secondary, and 
postsecondary school completion or the equivalent thereof, and (G) enrollment in other 
training programs or apprenticeships or enlistment in the Armed Forces. 

Section 111 of the Perkins Act identifies special populations and targeted groups. 
It also describes related data gathering and reporting requirements, several of which 
could be advantageously addressed in the statewide system of standards and measures. 
Data are to be gathered, analyzed, and disseminated on the adequacy and effectiveness 
of vocational education programs in meeting the education and employment needs of 
women (including displaced homemakers and single heads of households) and to ensure 
that needs of women and men for training in nontraditional jobs are met. States also are 
charged with identifying the number of students with handicaps enrolled in vocational 
programs and assuring that individuals with handicaps are receiving vocational education 
services. The number of economically disadvantaged students is to be identified and 
arrangements made to ensure that needs of such students are met. Grant recipients 
must also ensure that number of students with limited English proficiency is identified and 
that needs of such students for participation in vocational and applied technology 
education programs are being met. 

To ensure that students have opportunities to participate in programs that prepare 
them to enter employment. Section 111 instructs States to establish a limited number of 
technical committees to develop an inventory of skills needed for entry, retention, and 
advancement in occupational areas taught in the State. This inventory, which in Texas is 
maintained by the State Occupational Information Coordinating Committee (SOICC), is 
clearly germane to the de\ elopment and implementation of standards and measures 
regarding job or work skill attainment or enhancement 

Section 113 of the Perkins Act 
provides additional information about 
expected performance of vocational and 
applied technology education programs 
and, by implication, provides guidance for 
developing and implementing appropriate 
standards and measures of performance. 
Part (a)(3)(B) of this section says that 
vocational education programs should 
provide students with (i) strong 
experience in and understanding of all aspects of the industry the students are preparing 
to enter (including planning, management, finances, technical and production skills. 



Each recipient of financial assistance 
under part C cf title II shall annually 
evaluate the effectiveness cf the program 
conducted with assistance under this Act 
based on the standards and measures... 

Peridns Act cf 1990, Sec. 117 



underlying principles of technology, labor and community issues, and health, safety, and 
environmental issues; and (ii) strong development and use of problem-solving skills and 
basic and advanced academic skills (including skills in the areas of mathematics, reading, 
writing, science, and social studies) in a technological setting. 

Section 117 of the Perkins Act 
indicates how the system of statewide 
standards and measures of vocational and 
applied technology education programs is 
to be used. Each recipient of financial 
assistance for secondary vocational 
education programs under basic state 
grants for vocational education (Part C, 
Title II) is to annually evaluate the 
effectiveness of the programs conducted 
with assistance under the Act based on 
the standards and measures developed as 
required by Section 115. The annual 
evaluation is to (1) review programs, with 
the full and informed participation of 
representatives of individuals who are members of special populations to (A) identify 
and adopt strategies to overcome any barriers which are resulting in lower rates of access 
to vocational education programs or success in such programs for individuals who are 
members of special populations; (B) evaluate the progress of individuals who are 
members of special populations in vocational education programs assisted under the Act; 
and (2) evaluate the progress of vocational education programs assisted under the Act in 
providing students with strong e^qjerience in and understanding of all aspects of the 
industry the students are preparing to enter. If the evaluation shows that the recipient of 
financial assistance under the Act is not making progress in meeting the standards, the 
recipient is to develop a local plan, in consultation with teachers, parents, and students 
concerned, for program improvement for the succeeding school year. 

In order to meet the requirements for developing and implementing a statewide 
system of standards and measures of performance, the work plan for this project 
specified that 1990-1991 would be dedicated to conducting background research on 
performance standards and to delineating model performance standards for selected 
vocational and applied technology education programs. Objectives for 1991-1992 were to 
(1) conduct a feasibility study in 12 school districts of core performance standards 
developed during the first year of the project and (2) continue the development of 
standards for specific sequences of courses in vocational and applied technology 
education. During the summer of 1990, project staff initiated a literature review and a 
mailed survey of other state education agencies in order to develop a knowledge base 
from which to generate content and format for standards and measures of performance 
for secondary programs. In the fall of 1990, a General Advisory Committee was 



Based on the standards and measures, 
school districts shall: (A) identify and 
adopt strate^es to overcome barriers for 
individuais who are members of special 
populations; (B) evaluate progress of 
individuals who are members of special 
populations; and (C) evaluate progress in 
providing students with strong experience 
in and understanding of all aspects cfthe 
industry they are preparing to enter. 

Perkins Act cf 1990, Sec. 117 




convened to oversee and guide the project. This committee, constituted of 
representatives from private and public industry, school districts, postsecondary 
institutions, the State council on vocational education, and directors of vocational and 
applied technology education programs at the TEA, determined that the standards 
should be developed in two tiers: (1) "core" standards to address performance issues 
relevant across programs and (2) "course specific" standards to supplement the core as 
appropriate. At the close of the first year, the project had produced an annotated 
bibliography of key reference materials, a briefing on results of the survey of other state 
education agencies, and a preliminary draft of core standards. As these initial products 
were being developed, other important things were happening that influenced the 
direction and shape of this project. 



Expectations for America 2000: An Education Strategy 




In April 1991, the President of the United States announced AMERICA 2000: An 
Education Strategy to move communities toward achievement of National Education 
Goals adopted by the President and the Governors in 1990. Increased accountability is 
one part of the strategy with specific emphasis on development of standards that 
"incorporate both knowledge and skills to ensure that, when they leave school, young 
Americans are prepared for further study 
and the work force."^ A new nationwide 
examination system will be tied to the 
standards in English, mathematics, 
science, history, and geography. Congress 
will be asked to authorize a National 
Assessment of Educational Progress to 
collect state-level data in grades four, 
eight, and twelve beginning in 1994. 
Colleges will be urged to use the 
American Achievement Tests in 
admissions and employers will be urged to 
pay attention to them in hiring. 



Standards wUl be developed...[to] 
represent what young Americans need to 
know and be able to do if they are to 
live and work successfully in todays 
world. 

AMERICA 2000: An Educadon Strategy 




Secretary's Commission on Achieving Necessary Skills (SCANS) 

In support of the education strategy for America 2000, the U.S. Department of 
Labor published in June 1991 the first of a series of reports from the Secretary's 
Commission on Achieving Necessary Skills (SCANS)."^ The commission is charged with 
examining what work requires of schools with specific attention to (1) defining skills 
required to enter employment, (2) proposing acceptable levels of proficiency, (3) 
suggesting effective ways to assess proficiency, and (4) developing a dissemination 
strategy for the nation's schools, businesses, and homes. In the first report, the SCANS 
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presented its findings regarding skills needed for employment and presented a proposal 
for acceptable levels of proficiency. They concluded from their investigations that 
workplace know-how has two elements: competencies and ?i foundation. 

The competencies, now commonly referred to as SCANS competencies, include 
(1) Resources - allocating time, money, materials, space, and staff; (2) Interpersonal 
Skills - working in teams, teaching others, serving customers, leading, negotiating, and 
working well with people from culturally diverse backgrounds; (3) Information - 
acquiring and evaluating data, organizing and maintaining files, interpreting and 
conmiunicating, and using computers to process information; (4) Systems - 
understanding social, organizational, and technological systems, monitoring and 
correcting performance, and designing or improving systems; and (5) Technology - 
selecting equipment and tools, appljdng technology to specific tasks, and maintaining and 
troubleshooting technologies. 

The foundation for workplace know-how includes (1) Basic Skills - reading, 
writing, arithmetic and mathematics, 
speaking, and listening; (2) Thinking 
Skills - thinking creatively, making 
decisions, solving problems, seeing things 
in the mind*s eye, knowing how to learn, 
and reasoning; and (3) Personal Qualities 
- individual responsibility, self-esteem, 
sociability, self-management, and integrity. 

The proficiency scale proposed by 
the SCANS for workplace know-how has 
as its lowest level the designation of 
"preparatory" (suitable only for unskilled 
work). Succeeding levels are Vork 
ready," "intermediate," "advanced," and "specialist" (suitable for jobs requiring special 
expertise). Proficiency levels for the SCANS competencies would be assessed in grades 8 
and 12. The aim of these proposals, according to the SCANS, is to promote the 
development and use of assessment strategies that can provide the basis for a new kind 
of high school credential certifying that the competencies have been acquired. 

These recommendations build on those made by the authors of America's Choice: 
High Skills or Low Wages.^ In order to broaden the options for enabling young people 
to achieve the high skills needed in the workplace, it was recommended tiiat standards 
be established to award technical and professional certificates for occupations that do not 
require a baccalaureate degree. A "Certificate of Initial Mastery" would be awarded 
when the student demonstrates mastery of foundation skills and qualities (e.g., attaining 
the standards set for the core subjects American Achievement Tests proposed for 
America 2000). When they have mastered SCANS competencies and specific 



Workplace know-how has two parts: 
competence and a foundation cf skills 
and personal quaUties...This know-how 
will be important to those who will be 
developing the World Class Standards for 
education performance called f or. Jn 
AMERICA 2000. 

SCANS Report, June 1991 





occupational skills, students can be certified "work ready." Students who complete the 
base certifications early in their schooling can start working on advanced certifications- 
either in schools and colleges or in workplace training sites. 

The SCANS has committed to working with the U. S. Departments of Labor and 
Education to spearhead a public-private partnership to develop voluntary standards for 
all industries and link them to secondary school-based instruction, assessment, and 
certification of competencies as is called for in the America 2000 education strategy. 
Preliminary work toward this end already has been completed. In the report entitled 
Skills and Tasks for Jobs published by the SCANS in Februair 1992, the Commission 
identifies SCANS competencies and foundations for 50 jobs.^ The report describes how 
the skills and tasks were determined and provides examples of how the prioritized lists 
can be used in developing high school curricula to enable students to achieve necessary 
skills. 

In its final report. Learning a Living: A Blueprint for High Performance, the 
SCANS emphasizes the need for a rigorous performance-based assessment system. The 
Commission recommends that the system be education-based and nation-wide to permit 
educational institutions to certify levels of proficiency in workplace know-how that their 
students have achieved. The Commission did not, however, make recommendations 
about specific content or format for student assessment. 



Quality Work Force Planning in Texas 

Concurrent with the national movement toward cooperative endeavor to reform 
public education, new policies and programs are being established in Texas to (1) ensure 
excellence and equity in education, and 




(2) guarantee that the local work force 
has the high levels of skill thai are 
demanded for productivity to the year 
2000 and beyond. These actions have 
direct impact on the effort to develop and 
implement a statewide system of 
standards and measures of performance 
for vocational and applied technology 
education in secondary schools. 

In September 1990, a document 
entitled Texas Quality Work Force 

Planning was published to inform the public and to serve as a guide for a tri-agency 
initiative by TEA, the Texas Higher Education Coordinating Board, and the Texas 
Department of Commerce to initiate and manage cooperative planning at local and state 
levels.^ This program provides a crucial link between identifying employer needs and 



Quality Work Force Planning is the 
cmcial link that must be established 
between identifying employer needs and 
ensuring that indi\nduals who corrrphte 
vocational-technical education and 
training prop-ams are prepared for 
employment in the year 2000 and 
beyond. 

Tn-agpncy Initiative, September 1990 
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ensuring that individuals who complete vocational-technical education and training 
programs are prepared for employment in the year 2000 and beyond. The anticipated 
outcome is an integrated service delivery system that promot(» cooperation, 
communication, and the sharing of ideas, methods, and resources to (1) link education 
and training with regional needs of business and industry, (2) reinforce economic 
development by producing a quality work force; (3) increase the efficiency and cost 
effectiveness of education and training by matching programs to changes in labor market 
demands; and (4) encourage flexible and responsive delivery systems which will provide 
equal access for the region's population. The State Plan for Vocational and Applied 
Technology Education for Fiscal Years 1992-1994 cites the activities of regional Quality 
Work Force Planning Committees as means for increasing efficiency and cost 
effectiveness of delivering occupational education and training for jobs that are and will 
be available in Texas.^ 




Development of the Academic Excellence Indicator System (AEIS) 

In January 1991 a specially appointed advisory committee submitted its 
recommendations of a set of academic excellence indicators to the State Board of 
Education. The Board is directed in sections 21.753 and 21.7531 of the Texas Education 
Code to adopt a set of indicators of the 
quality of learning on a campus and other 
performance standards for purposes of 
evaluation, accreditation, and 
determination of exemplary status. 
Paragraph 17 of section 21.753 says that 
the criteria for evaluating performance of 
school districts must address the quality 
and effectiveness of the districts' 
vocational education programs. 



The State Board of Ectucation..shall 
adopt a set of indicators of the quality of 
learning on a campus and other 
performance standards... 

Texas Educadon Code, Section 21J531 




The set of indicators adopted by the Board in 1991 are (1) results of criterion- 
referenced test of academic skills; (2) results of norm-referenced test of academic skills; 
(3) results of academic skills test administered to students entering college in Texas; (4) 
results on Scholastic Aptitude Test and American College Test; (5) one-year high school 
graduation rate; (6) enrollment in advanced courses; (7) student attendance; (8) dropout 
rate; (9) percent of seniors receiving "advanced" or "advanced with honors" seals affixed 
to transcripts; and (10) post-graduate performance in the workplace. 

In July 1991 the Board requested assistance from the Texas Council on 
\ ocational Education in identifying criteria to measure the performance of high school 
graduates in the workplace. As a result of its study, the Council prepared six 
recommendations and presented them to the Board in a report published in October 
1991 entitled "Demanding Excellence, Rewarding Quality."^ These six 
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recommendations, which were accepted by the Board in February 1992, are (1) e5q>and 
the postgraduate performance indicator to include emplojinent status, earnings, 
postsecondary enrollments, completion of "2+2" programs, degrees, and certificates; (2) 
include a set of surveys which measure employer, postsecondary, and student satisfaction; 
(3) Inxlude longitudinal measurement of post-graduate performance; (4) take advantage 
of data base systems being used by other states (e.g.. Unemployment Insurance Wage 
Record System); (5) expand the indicators to include a measure of personal management 
skills and competencies; and (6) use Quality Work Force Planning Committees to 
conduct employer surveys and gather information not derived through a data base 
search. 

The Council also noted that standards for occupational skills are different for 
different occupations. Observing that the current indicators do not assess attainment of 
specific occupational skills, the Council proposed that "if completion of a "Tech Prep' 
program was added to the list of indicators, the quality of occupational preparation could 
be implied."-''^The list of indicators presented to the Board by the Commissioner of 
Education in January 1992 includes "Completion of Recommended Preparatory Program" 
and suggests as a measure "the percent of students who have completed board adopted 
preparatory programs." ■'••^ 



Student Assessment Plan for Schools in Texas 

The Committee on Student Learning was appointed by the legislature to study 
issues and make recommendations to the 
Board regarding the student assessment 
program in this state. In February 1992 
the Committee made the following 
recommendations which not only have 
relevance for the AEIS but also provide 
important guidance for development and 
implementation of standards and 
measures of performance for vocational 
and applied technology education: the 
assessment strategy should be (A) based 
on outcomes; and (B) primarily 

performance based, consisting of a combination of standardized paper and pencil 
measures and performance measures. ■'•^ 



Board Approved Priority Occupations for Texas 

During 1991 a task force reviewed labor market and work force planning 
information to update for the State Board of Education a list of jobs for which future 
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Assessment should be... 

* based on outcomes; 

* primarily petformance based... 
CommiUee on Student Learning, 1992 



demand in Texas is substantial, wages are forecast to be greater than $6.44 per hour, and 
training requirements do not exceed four postsecondary years. In March 1992, the Board 
approved a list of approximately 60 priority occupations. To this list of jobs was added a 
short list of additional, new, and emerging occupations that, although not meeting all of 
the review criteria, offer promising opportunities for students enrolled in schools in 
Texas to contribute to the state's economic growth. The approved list includes specific 
occupations that students could prepare to enter by enrolling in vocational and applied 
technology education programs in agriculture (e.g., production clerk), home economics 
(e.g., child care), health occupations (e.g., medical assistant), industrial technology 
(computer operator), marketing (e.g., sales person), office (e.g., legal secretary), or trade 
and industrial education (e.g., welder/cutter). 

Because one of the purposes of the system of standards and measures of 
performance demanded in the Perkins Act of 1990 is to ensure that vocational and 
applied technology education programs provide students with strong experience in and 
understanding of all aspects of the industry they are preparing to enter, the list of Board 
approved priority occupations has special relevance in this project. The importance of 
this list is further underscored in comments made to the Board by staff of the TEA in 
April 1992 pointing out that, ultimately, standards and measures should be developed not 
only to comply with federal law but more importantly to measure and encourage the 
pursuit of equity and excellence in public education. Standards should, therefore, 
address real world outcomes and appropriate measures be developed and validated to 
ensure that industry standards are met. An approved list of priority occupations provides 
focus for the development and implementation of a statewide system of standards and 
measures of performance for vocational and applied technology education in secondary 
schools in Texas. 





The overall objective of this project is to 
research and develop measurable and 
objective cnteria to assess the quality and 
performance cf vocational and appUed 
technology education in secondary 
schools in Texas: 

Objective A: Conduct a feasibility study 
of core performance 
standards in a sample of 
12 school districts in 
Texas. 

Objective B: Continue development of 
standards for specific 
sequences cf courses in 
vocational and applied 
technology education. 

Project Work Plan, July 1991 




Section 2: GOALS AND OBJECTIVES 

In order to meet the challenge of designing and implementing by July 1992 an 
evaluation system to meet the requirements of the Perkins Act of 1990 and enable school 
districts in Texas to document progress of 
secondary school students participating in 
vocational and applied technology 
education programs, TEA authorized 
research and development contract 
awards to the Extension Instruction and 
Materials Center (EIMC) at The 
University of Texas at Austin. The first 
award was made in July 1990 and the 
second in July 1991. The work plans for 
this sequence of two one-year projects 
was designed to research and develop 
measurable and objective criteria to assess 
the quality and performance of vocational 
and applied technology education in 
secondary schools in Texas. 

Specific objectives for project year 
1990-1991 were to (A) conduct 
background research on performance 
standards in vocational and applied 
technology education, and (B) delineate 
model performance standards for selected 
vocational programs. During 1990 and 
1991, library research was conducted, a 

mailed survey of other states was made, a general advisory committee was convened and 
consulted, a set of "core" standards was drafted, and a series of teleconferences were 
conducted with small, specially assembled advisory groups to review, critique, and revise 
the proposed standards. In September 1991, the person who was the project manager 
during 1990-1991 resigned from the project for health reasons. An entirely new research 
and development team was brought into the project in October 1991. The final report 
for project year 1990-1991 which describes activities and products of the first year's 
efforts was prepared from project records bv the new project manager. That report was 
submitted to tlie TEA in December 1991.^"^ 

Objectives for project year 1991-1992 were to (A) conduct a feasibility study of 
core performance standards in a sample of 12 school districts in Texas and (B) continue 
the development of model performance standards for specific sequences of courses in 
vocational and applied technology education. The work plan for this second and final 
year of the project states that three products will be delivered: (1) course-specific 
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performance standards for those vocational courses for which they are needed; (2) a 
feasibility study of the core performance standards developed during 1990-1991; and (3) 
a data-gathering instrument that vocational administrators in local school districts can 
use in conducting their districts' annual evaluations based on the core standards and 
measures of performance. A copy of the abstract of the work plan for project year 1991- 
1992 is in Appendix A. 
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Section 3. PROJECT DESIGN AND PROCEDURES 

This project used different procedures to achieve the two objectives. Objective A 
was addressed by visiting 12 school districts to interview school district personnel and 
pilot test the core performance standards, measures of performance, and prototype data 
collection instrument (see Table 1). To attain Objective B a multi-attribute policy 
analysis procedure was used to evaluate the options and formulate recommendations 
regarding standards for specific sequences of courses in vocational and applied 
technology education (see Table 2). 



Table 1: Overview of Procedures Used to Address Objective A 



OBJECTIVE 



PROCEDURES 



Objective A: Conduct a 
feasibility study of core 
performance standards in a 
sample of 12 school 
districts in Texas 




Drafts of performance standards and prototype 
data collection instrument were submitted to 
the General Advisory Committee for critique; 

Interviews, records reviews, and piloting of the 
data collection instrument took place in 12 
schools districts; and 

Feasibility was assessed against criteria 
established by the General Advisory 
Committee. 




Preparing Core Standards for Feasibility Study 

In October 1991, the General Advisory Committee for this project met in Austin 
to review and revise drafts of core performance standards that were developed during 
project year 1990-1991. Section 1.7 of the state plan for vocational and applied 
technology education for 1992-1994 says that three standards will be implemented for 
school year 1992: (1) measures of learning and competency gains; (2) job or work skill 
attainment or enhancement; and (3) placement into additional training or education, 
military service, or employment. As is noted on page 1 of this document, the Perkins 
Act of 1990 requires only two standards and measures of performance. The Advisory 
Committee and the project team during 1990-1991 were more comprehensive in their 
approach and identified one or more standards and measures to address each component 
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and element named in Section 117 (b) (Requirements) of the Act. Drafts that were 
presented at the meeting of the committee in October of 1991 included, therefore, a 
total of 11 "core" standards and measures of performance that would be applicable across 
all vocational and applied technology education programs in secondary schools in Texas. 

The meeting was attended by one representative from industry, two from school 
districts, two from postsecondary schools, two from the state council on vocational 
education, and nine program directors of vocational and applied technology education 
from the TEA. The committee recommended substantial revision of one of the 11 
standards and changes in wording for 8 others. The committee also expressed concern 
about what constitutes a "coherent sequence" of courses in a vocational and applied 
technology education program. Representatives from the state council on vocational 
education informed the committee that their report to the Board recommends that a 
student who concentrates in vocational education should be defined as an individual who 
has successfully earned at least four credits in an occupational field. It was suggested 
that the issue will require longitudinal research for its resolution. Despite these 
concerns and in the interest of being timely in implementing the statewide system that 
the Perkins Act requires, the advisory committee approved plans for the feasibility 
assessment to be conducted in November and December of 1991. A copy of the minutes 
of this meeting of the General Advisory Committee is in Appendix B. A copy of the 
standards and measures as revised and approved for field assessment also is included in 
Appendix B. 



Visiting School Districts to Collect Data 

Participation in the feasibility study was recruited to represent districts of different 
sizes located in different parts of the 
state: central, east, far south, far west, 
north, and north central. The vocational 
administrator in these districts received an 
advance copy of the proposed standards 
and a draft of the data collection 
instrument. The administrators were 
asked to use data they already had on 
hand for school year 1990-1991 to fill in 
as much of the data collection instrument as possible prior to the arrival of the project 
evaluation staff. 

Single day trips to the 12 school districts were made in November and December 
1991. Project staff usually were on site from 9:00 am to 4:00 pm. The morning hours 
were spent meeting with the vocational program administrator and other school district 
personnel invited by the administrator. These meetings usually were attended by a 
guidance counselor, special populations educator or coordinator, and one or more 
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Four large, 3 medium size, 4 small, and 1 
very small school districts participated in 
the field study in November and December 
1991. 
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persons from the district's management information office or unit The majority of the 
meeting time was spent discussing possibilities for obtaining information called for on the 
data collection instrument in order to measure program performance. Having 
determined what data bases and records should be consulted, school district personnel 
located as many of the records and sources of information as possible at that time. 
Project personnel spent the afternoon hours compiling the information onto the data 
collection instrument. An exit interview was conducted to review results of the 
evaluation, determine how missing data might be obtained, and solicit critique and 
suggestions for revising the standards and restructuring the data gathering instrument. 



Assessing Feasibility of Core Standards 

The work plan for this second year of this project listed nine criteria for 
evaluating feasibility of the standards and measures of performance. These criteria, 
which were derived from guidance 
provided by the General Advisory 
Committee during its two meetings in 
1990-1991, specified that the standards 
and measures should be (1) clearly 
defined, (2) cost efficient, (3) manageable 
in number, (4) relatively easy to measure, 
(5) timely, (6) goal oriented, (7) 
attainable, (8) credible, and (9) focused 
on product rather than process. 
Information obtained during the on-site interviews and from the process of entering data 
onto the data gathering instrument was used to determine the extent to which the 
standards and measures met these feasibility criteria. 

A summary report was drafted in mid-January 1992 and submitted to program 
directors at the TEA. In a meeting with this subgroup of the General Advisory 
Committee, additional critique and suggestions for increasing the usability of the 
standards and of the data gathering instrument were obtained. 



The standards should be cleady defined, 
cost efficient, manageable in number, 
reladvely easy to measure, timely, goal 
oriented, attainable, credible, and focused 
on product rather than process. 




Identifying Options for Course-Specific Standards 

One of the major activities of project year 1990-1991 was a review of the 
literature and the production of an annotated bibliography regarding the state of the art 
and science of evaluating performance of vocational and applied technology education. 
The materials assembled in that effort were very useful in determining what the options 
are for developing course-specific performance standards. In addition to reviewing 
written materials, project staff met several times with staff at the TEA and consulted 
Individuals at the Center for Human Resource Development at the University of Texas, 
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at the Private Industry Council of Austin/Travis County, and at the Texas Department of 
Commerce regarding possible strategies for developing standards and measuring rtudent 
outcomes for specific courses. 



Table 2: Overview of Procedures to Address Objective B 



• 





PROrFDlIRES 


B Continue 


* Information collected during 1990-1991 and 


development of 


suggestions m^de by school district personnel 


standards for 


during the feasibility assessment were reviewed 


specific sequences 


and guidance was sought from program 


of courses in 


directors at the TEA to identify options for 


vocational and 


developing course-specific standards; 


applied technology 


* Cost-and-Benefit criteria were developed for 


education 




evaluating the options; 




* Options were compared and the most promising 




option selected and used to formulate 




recommendations regarding standards and 




measures of performance for at least one 




specific course in each program area. 



Criteria for Comparing the Options 

A list was made of statements from the SCANS reports, minutes of meetings of 
the State Board of Education, and minutes of the General Advisory Committee about 
characteristics that standards and measures of performance should have. This list, which 
is shown in Table 3, was used as a guide in comparing the benefits of the different 
options identified for development of course-specific standards. 

The first criterion is that the standards be based on outcomes. This was a specific 
recommendation of the Committee on Student Learning appointed by the Texas 
Legislature. An outcome is "a culminating demonstration of prior learning that is 
applied in a specified performance context."-^'* Outcomes as they apply to specific 
courses of study in vocational and applied technology education in secondary schools in 
Texas means that students are prepared to get and keep jobs that are and will be 
available in the local economy. 
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Table 3: Dimensions Used to Compare Options for Course-Specific Standards 



Course-Specific Standards and Measures of Performance Should... 

1 be based on outcomes; 

2 assess attainment of specific occupational skills; 

3 assess levels of proficiency for SCANS competencies; 

4 assess use of problem-solving and basic and advanced academic skills; 

5 be primarily performance-based; 

6 be education-based and nation-wide; and 

7 apply to all students pursuing courses of vocational and applied technology 
education, including those who are members of special populations. 




Cost Considerations in Development and Implementation of Course-Specific Standards 
include... 



1 how soon the system will be available; 

2 how much instructional time would be consumed in administering the system; 

3 dollar costs of system development/acquisition; and 

4 dollar costs of system operations and maintenance. 



The second criterion is that the system assess specific occupational skills. This 
recommendation is derived from the Perkins Act. In particular, the Perkins Act 
indicates that assessment should address students' understanding of all aspects of the 
industry they are preparing to enter including planning, management, finances, technical 
and production skills, underlying principles of technology, labor and community issues, 
and health, safety, and environmental issues. This recommendation is reinforced in 
JTPA which names as a performance indicator for job training programs for youth 
"attainment of recognized employment competencies." 




The third criterion for evaluating alternative strategies for development and 
implementation' of course^pecific standards and measures of performance is capabilities 
for assessing levels of proficiency in the SCANS competencies. The SCANS believes 
that all students should be able to demonstrate these competencies by the time they can 
legally leave high school, age 16 in most states. A detailed listing of the competencies is 
in A^jpendix D. 

The fourth criterion is that, in addition to specific occupational skills and SCANS 
competencies, standards and measures should also assess use of problem-solving and 
basic and advanced academic skills in a technological setting. The Perkins Act specifies 
that these skills should include mathematics, reading, writing, science and social studies- 
This requirement is convergent with the stipulation in America 2000: An Educadon 
Strategy that standards incorporate both knowledge and skills to ensure that, when they 
leave school, young Americans are prepared for further study and the workforce. The 
America 2000 strategy recommends that assessments in grades 4, 8 and 12 address 
student progress in English, mathematics, science, history, and geography. The SCANS 
recognizes the importance of these kinds of basic skills and qualities and lists them 
among the elements that are the Foundation of workplace know-how. 

The fifth criterion is that the system be primarily performance based, consisting of 
a combination of standardized paper and pencil measures and performance measures. 
This recommendation was made to the State Board of Education by the Committee on 
Student Learning. It helps to define what is encompassed in the concept of assessing 
basic skills in a technological setting. It also is consistent with the observation of the 
SCANS that most existing tests-largely pencil and paper, multiple-choice tests of short- 
term memory-do little to advance the cause of learning. Performance based assessment 
is a valuable tool not only as a means for certifying the att?inment of competencies but 
also as a guide for the learner and the instructor about how to distribute future learning 
time to enable mastery. The final SCANS report emphasizes, therefore, the need for 
rigorous performance-based assessment. 

The sixth criterion is that the system be education-based and nationwide. This 
recommendation is made in the final report of the SCANS. The system should permit 
educational institutions to certify levels of proficiency in workplace know-how that their 
students have achieved. It is e?q>ected that colleges would use the results in admissions 
and employers would pay attention to them in hiring. The aim is to provide a basis for a 
new kind of high school credential certifying that competencies have been acquired. By 
making the system education-based and nation-wide, progress will be made toward 
establishing standards to award technical and professional certificates to ensure quality in 
the work force and facilitate career progression in the wide variety of occupations that 
do not require a baccalaureate degree. 

The seventh and final "benefit" criterion for comparing options for course-specific 
standards in vocational and applied technology education is the extent to which the 
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S3/stem is useful in evaluating progress of individuals who are members of special 
populations- The Perkins Act requires tliat the statewide system of core standards and 
measures be useful in this regard. It is also reasonable to demand that course-specific 
standards lend themselves to this purpose so as to ensure access and success in school 
and in the work force for persons who are members of special populations. 

In addition to these seven "benefits" dimensions. Table 3 also shows a short list of 
four considerations that were used to estimate and compare costs of the different 
approaches. The first consideration is how soon the standards and measures will be 
available for integration into practice. The SCANS took two years to study the problem 
and make its recommendations and is, at the end of its effort, accused of "ducking major 
issues" regarding standards and measures of performance.*'-^ Similarly, the National 
Advisory Commission on Work-Based Learning, a permanent group appointed by the 
Secretary of Labor to make recommendations regarding industry-wide, portable skill 
standards, is reported to have run head-on into obstacles that are thwarting its 
progress.-^ ^ Apparently, development of standards takes much longer and/or is more 
difficult than is forecast. Timeliness appears, therefore, to be a critical issue in deciding 
which option is best for development of course specific standards for vocational and 
applied technology education. 

In documents outlining development of the AEIS in Texas, as well as in the 
SCANS reports, concerns frequently are expressed regarding the burden of time that is 
placed on the learning environment by testing and other forms of student assessment 
The AEIS is being developed to increase efficiency so that the frequency of testing can 
be minimized. The SCANS advocates use of assessment strategies that contribute to 
learning so that assessment time is instructional time. Amount of time demanded for 
student assessment remains an area of concern and is, therefore, the second "costs" 
consideration for comparing options for course-specific standards. 

The third and fourth considerations are dollar costs. Two kinds of costs were 
estimated and evaluated. Development and acquisition costs refer to all costs incurred 
before the system can be integrated into practice-i.e., the pre-implementation costs. 
Operations and maintenance costs refer to those dollars that must be spent to administer 
the assessments and report results. They also include the post-implementation research 
and development costs that are incurred in keeping the assessment system contemporary 
with emerging technology. As requirements in the workplace change, assessment of 
student progress toward attaining competencies that position them to be productive in 
the workplace also must change. The assessment system must have built into it a 
strategy for keeping the standards and measures up-to-date so that colleges wiU use 
results in admissions and employers will pay attention to them in hiring. The costs of 
keeping the system current must be considered "up-front" in deciding which option is best 
for development and implementation of course-specific standards in vocational and 
applied technology education. 
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Fonnulating Recommendations Regarding Course-Specific Standards 



The work plan for project year 1991-1992 indicated that course-specific standards 
would be considered for "one sequence of two to four courses within each program area." 
The minutes of the General Advisory Committee meeting of October 1990 included a list 
of two or three specific courses in each program area that the committee had nominated 
as candidates for the development of course-specific standards. The idea of using this 
list as a guide in this final phase of the project was abandoned, however, because it is 
inconsistent with the "design down" approach advocated for outcome based education. 
Unless there is assurance that the course itself was "designed down" from real-world 
requirements or industry standards, one cannot develop an assessment system that is 
based on outcomes by starting with a list of courses or sequences of courses. Designing 
down requires that one begin with a list of jobs that todays students will fill tomorrow 
and then work backward from industry standards to develop the system for assessing the 
student's progress toward the real world outcomes of getting and keeping those jobs. 
For that reason, the list of State Board of Education approved priority and emerging 
occupations was used as the starting place for formulating course-specific standards in 
vocational and applied technology education. 
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Section 4: RESULTS AND ACCOMPLISHMENTS 

Results and accomplishments are discussed in two parts. The first part of this 
section of this report addresr^ Objective A: Feasibility Study of Core Standards and the 
second part addresses Objective B: Development of Course-Specific Standards. An 
overall summary of findings is shown in Table 4. 

Table 4; Summary of Findings 
Regarding results of the feasibility study for core standards- 

* School district personnel were accepting of the number and content of the standards. 

* More guidance aJid definition was requested regarding (1) what constitutes a 
"coherent sequence of courses;" (2) strategies for compiling data for the evaluation; 
and (3) how lo use the information to develop local program improvement plans. 

* Results ol the study were used to refine and clarify the standards, measures of 
performance, and prototype data collection instrument, which were submitted to TEA. 

* The set of three standards finally approved by both the Committee of Practitioners 
and the State Board of Education for school year 1992-1993 were addressed to: (1) 
passing the statewide academic skills exit test; (2) certification of occupational 
competencies; and (3) postsecondary placement. 



Regarding development of standards for specific courses... 

* The 22 Board approved priority or emerging occupations that were identified as jobs 
that a young person who is still in or has just graduated from high school could expect 
to enter given appropriate preparation and credentials were selected as targets for 
this project. 

* Of the three options identified as plausible programs for developing course-specific 
standards to address these 22 occupations, the Student Occupational Competency 
Achievement Test (SOCAT) was selected as offering the greatest benefits at the least 
costs. 



* Blocks of specific courses that students could take to master elements assessed in the 
SOCAT were identified for all 14 of the target occupations for which SOCATs are 
available and for an additional 8 approved occupations that usually require 
postsecondary training but do not require formal apprenticeship, baccalaureate, or age 
and experience. 
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Table 5 is taken from the interim report that was submitted to the TEA in 
January 1992 and provides an overview of results of the feasibility study. 





Table 5: Overview of Results Relating to 
Objective A: Feasibility Study for Core Standards and Measures of Performance 

Clearly defined: In all 12 districts that participated in the study, concern was expressed 
regarding definition of "a coherent sequence of vocational education courses." 

Cost efficient: Personnel in all 12 districts indicated that the necessary information is 
available through the Public Education Information Management System (PEIMS), but 
new data processing programs will be needed to extract the data. 

Manageable in number: No one seriously objected to the number of standards (i.e., 11) 
but they did point out some duplication of effort - e.g., asking whether or not students 
passed the Texas Assessment of Academic Skills (TAAS) by the end of their 12th grade 
year and then asking whether or not students graduated generates almost identical data. 

Relatively easy to measure: Although only two of the 11 measures could be calculated 
during the site visits, district personnel indicated that measurement would be easy if 
programs were written to pull information from PEIMS and if an efficient strategy was 
developed to identify students enrolled in "a coherent sequence of courses." 

Timely: Because the standards address events that occur at different times in the year - 
i.e., enrollment at the beginning of the year, promotion or graduation at the end of the 
year, and placement after graduation - school district personnel requested guidance 
about when to conduct the evaluation. 

Goal Oriented: The majority of program administrators in the 12 districts reported that 
the information requested is the kind of performance data they would like to have in 
order to keep their programs working as well as possible. 

Attainable and Credible: In the one very small district where it was possible to gather 
all data needed to complete the entire evaluation (because it was possible to pull records 
for all 30 of the students who were enrolled in vocational education courses), all 
standards were met except for the one regarding linking course offerings to Quality 
Work Force Planning. 

Focused on outcomes rather than process. District personnel made several suggestions 
about how to revise the standards to give more attention to outcome - e.g., enquiring 
about enrollment in programs relative to work force outlook rather than asking if the 
outlook was used in making decisions about course ofi'erings. 
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Feasibility of Core Standards and Measures of Performance 

Table 5 provides an evaluation of the proposed standards and measures against 
the nine criteria that were named in the work plan as characteristics of a feasible and 
useful system. The most frequent request made by district personnel during the 
feasibility study was for more specific guidance regarding the definition of "coherent 
sequence of courses." Their other recommendations for improving the proposed core 
standards and measures of performance were (1) The data collection form should be 
structured to provided notation regarding which pieces of data should be sought from the 
Public Education Information Management System (PEIMS) and for data not found in 
PEIMS notations should be made as to where these data can be found; (2) Specific 
instructions should be provided to guide the calculation of each measure of performance; 
(3) The data collection instrument should be structured to integrate the standards, 
suggested data sources, and calculation steps into a single document; (4) The number 
of standards should be reduced by combining those that provide essentially the same 
information; and (5) Layout of the data gathering instrument should provide a "map" to 
guide the local evaluator through the data gathering and data evaluation steps. These 
suggestions guided revisions of the standards and of the data gathering instrument. By 
combining those that duplicated one another, the number of standards was reduced from 
11 to 8. 

The revised core standards, which are listed in Table 6, were presented to school 
district personnel at the Mid-Winter Vocational Supervisors, Administrators, and 
Counselors' conference. They were detailed in the report submitted to the TEA in 
January 1992. The report was reviewed by staff at the TEA and forwarded in February 
1992 to the state appointed Committee of Practitioners that, in accord with requirements 
of the Perkins Act, was charged with reviewing the proposed standards and measures 
before they are presented to the Board and integrated into rules and regulations. The 
Committee of Practitioners (1) revised the first standard to specify that performance on 
the Texas Assessment of Academic Skills (TAAS) secondary school exit exam would be 
the measure of performance rather than promotion to the next grade level, (2) elevated 
the criterion values for the standards to specify 95 percent of students should pass the 
TAAS exit exam and 95 percent should graduate from high school, and (3) made 
wording changes in several others of the standards. Further review of the proposed 
standards by staff at the TEA, however, resulted in substantial revision of the standards. 







Table 6: Core Standards Recommended based on Results of Feasibility Study 

1 At least 80 percent of students in grades 9 through 11 who were enrolled in a coherent 
sequence of courses in vocational education programs were promoted to the next 
grade leveL 

2 At least 80 percent of 12th grade students who were enrolled in a coherent sequence 
of courses in vocational education programs that included, where applicable, at least 
one occupationally specific course (a) obtained a certification of competency by an 
accepted licensing or certification agency, OR (b) successfully completed a validated 
test of occupational competency; OR (c) demonstrated completion and competency in 
the essential elements for the coherent sequence of courses* 

3 The school dropout rate for students in grades 9 through 12 who were enrolled in a 
coherent sequence of courses in vocational education programs was no greater than 
the school dropout rate for secondary students district-wide. 

4 Among 12th grade students enrolled in a coherent sequence of courses in vocational 
education programs that included, where applicable, at least one occupationally 
specific course, the percentage who obtained high school diplomas was comparable to 
the percent of all 12th grade students in the district who earned high school diplomas. 

5 At the one-year follow-up, 75 percent of students who earned their high school 
diploma after having completed a coherent sequence of courses in a vocational 
education program that included, where applicable, at least one occupationally ^)ecific 
course were (a) enrolled in a post-secondary educational institution; OR (b) enrolled 
in a registered apprenticeship program; OR (c) in training related to their vocational 
education program; OR (d) in military service; OR (e) employed in a paid or unpaid 
job related to their vocational education and training. 

6 Of students in grades 9 through 12 who are members of special populations, the 
percentages enrolled in vocational education courses was comparable to percentages of 
those groups of students within the school district. 

7 Members of special populations who received special or supplemental services because 
they were enrolled in vocational education programs were (a) promoted to the next 
grade level; OR (b) obtained high school diplomas at rates that were greater than 
those achieved by comparable groups of students who were not receiving special or 
supplemental services from vocational education. 

8 Enrollment in vocational education programs was consistent with the regional (or 
statewide) occupational outlook compiled as part of the quality work force program in 
the state of Texas. 
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In April 1992, a very reduced set of standards was prepared by staff at the TEA 
and resubmitted to tlie Committee of Practitioners. Table 7 lists the three standards 
finally approved both by the Committee of Practitioners and by the State Board of 
Education in May 1992 for school year 1992-1993. These three standards are expansions 
of those adumbrated in Section 1.7 of the state plan for vocational and applied 
technology education for 1992-1994. The prototype data collection instrument has been 
revised to address just these three standards. The instrument is in Appendix D. 
Additional information that TEA staff prepared to support their recommendation that 
these three standards be adopted for school year 1992-1993 noted that "other indicators 
relating to program effectiveness must be developed."^^ Staff also proposed that follow- 
up conducted four or five years after graduation may be more appropriate than the one- 
year follow-up required in standard number three. Their written comments also noted 
that "appropriate performance tests must be developed and validated, to ensure that 
industry standards are being met." This last recommendation is annotated in standard 
number two and is tightly linked to this project's Objective B, which is to develop 
standards for specific sequences of courses. 



Table 7: Core Standards and Measures Approved by the Board for 1992-1993 




At least 90 percent of students who were enrolled in a coherent sequence of 
courses in vocational and applied technology education will pass the TAAS exit 
test. 

At least 95 percent of 12th grade students who were enrolled in a coherent 
sequence of courses in vocational and applied technology education programs (a) 
obtained a certification of competency by an accepted licensing or certification 
agency, OR (b) successfully completed a validated test of occupational 
competency, OR (c) demonstrated completion and competency in the essential 
elements for the coherent sequence of courses. Within three years this standard 
will be based on performance measures which assess the level of proficiency 
required by employers and/or institutions of higher education. 

At the one-year follow-up, 75 percent of students who earned their high school 
diploma after having completed a coherent sequence of courses in a vocational 
and applied technology education program were (a) enrolled in a postsecondary 
educational institution, OR (b) enrolled in a registered apprenticeship program, 
OR (c) in training related to their vocational education program, OR (d) in a 
military service, OR (e) employed in a paid or unpaid job related to their 
vocational and applied technology education and training. 
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Table 8 provides an overview of findings regarding development of course-specific 
standards. 

Table 8: Overview of Results Relating to 
Objective B: Deve lopment of Standards for Specific Sequences of Courses 

* Priority Occupations. Of the approximately 70 jobs that are approved as priority 
or emerging occupations by the State Board of Education in Texas, 15 require a 
baccalaureate degree, another 13 require that the occupant be at least 21 years 
old and/or are not entry level jobs, 5 are apprenticeship occupations, 13 others 
usually require some postsecondary training, leaving 22 occupations that a young 
person who is still in or has just graduated from high school could expect to enter 
given the appropriate preparation. 

* Options for Developing Standards for Specific Courses Linked to the Priority 
Occupations. Three options were identified as plausible programs for 
development and implementation of course-specific standards to assure that 
industry standards are met: (1) validate competency profiles that school districts 
currently are required to maintain for each student; (2) use assessment tools that 
already are developed and being used in other vocational and applied technology 
education programs across the nation; and (3) design, develop, implement, and 
maintain a new system. Use of the SOCATs currently available from NOCTI was 
identified as the option with the greatest relative advantage. 

.* Cross-walking SOCATs to Priority Occupations and Specific Courses. SOCATs 
already are available for 14 of the 22 priority occupations that young persons can 
enter while still in or just graduated from high school and for 8 of the 13 other 
approved priority occupations that do not require apprenticeship or baccalaureate 
degree. Sets of two to four courses in the current curriculum for vocational and 
applied technology education in secondary schools in Texas were identified as 
specific courses that would prepare the student to demonstrate mastery of 
knowledge and skills assessed with these 22 SOCATs. 



Occupations from which to "Design Down" to Course-Specific Standards 

The Occupational Outlook Handbook published by the U.S. Department of Labor 
was consulted for information with which to sort the Board approved priority and 
emerging occupations into categories that reflect the amount of training that e>^rts 
believe is needed to make the job candidate "work ready" for that occupation. Results 
of this analysis are snown in Table 9. 
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Table 9: Jobs Approved by the State Board of Education 
as Priority or Emerging Occupations in Texas 




Occupational skills training not required ; 

dental assistants 

gardeners/groundskeepers 

receptionists 

salespersons* parts 

stock clerks, sales floor 

Occupational skills training an advantage : 

bilUng clerks 

carpenter's helpers 

diesel mechanics 

electrician's helpers 

legal secretaries 

maintenance repairers 

mechanic's helpers 

medical assistants 

medical secretaries 

other construction helpers 

other health service workers 

other protective service workers 

other secretaries 

plumber's helpers 

production clerks . 

welders & cutters 

word processing typists 

Postsecondarv training/college preferred : 

auto mechanics 

child care woilcers 

computer operators 

computer programmers/aides 

drafters 

machinists 

radiologic technicians 
electrical/electronic technicians 
food service managers 
licensed practical nurses 
other engineering technicians 
management support woricers 
preschool teachers 



A pprenticeship preferred /required : 
caipenters 

electric powerline installers 
electricians 

painters/paper hangers 
plumbers/pipefitters 

Baccalaureate preferred /required : 

adjustment clerics 

bill and account collectors 

business service agents 

elementary teachers 

insurance sales persons 

other managers 

other teachers 

personnel managers 

personnel ^cialists 

property/real estate managers 

registered nurses 

sales representatives, \^4iolesale 

social woricers 

special education teachers 

systems analyst, EDP 



Not entry level iob OR occupant must be 21 4-: 
corrections officers 
first line supervisors: 
clerical 

construction trades 

mechanics 

production 

sales 

service 

other 

industrial truck operators 
police patrol officers 
school bus drivers 
tellers 

truck drivers 




A worksheet included in Appendix E provides summary comments to note why 
the jobs were categorized as shown in Table 9. The worksheet explains, for example, 
that entries in the Occupational Outlook Handbook under the heading of •'dental 
assistants," indicate that employers seek persons with a congenial personality to fill these 
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positions. Information under the heading of "billing clerks" indicates that the work 
usually requires substantial on-the-job-training. And maintenance repair worker is a 
term that covers a number of specific occupations. One of these is home entertainment 
equipment repairer. The worksheet shows this type of maintenance repairer usually is 
self-employed. 

It should be noted rliat some of the specific occupations that have lesser training 
requirements probably serve as entry positions on career ladders for other occupations in 
the approved list that demand greater amounts of training. The position of stock clerk, 
for example, has no formal training requirement. It could serve, however, as a first step 
toward becoming a first line sales supervisor, a position that usually is filled by persons 
with substantial experience in the industry. This kind of progression is especially 
noteworthy for the positions that require formal apprenticeship. In many regions of 
Texas today these programs are concentrated in construction trades where the number of 
apprenticeship slots is shrinking and generally filled from "within" the industry. In 
order to enter an apprenticeship program to progress in the carpentry trade, for example, 
it may be necessary to begin as a carpenter's helper. 

The list of occupations categorized on the basis of amount of training that ejqierts 
indicate would be preferred or required to make a person "work ready" provided the 
base around which to identify and compare options for developing standards for specific 
courses in vocational and applied technology education. Thirteen of the 67 jobs listed in 
Table 9 require that the occupant be at least 21 years old (e.g., police patrol officers 
must be at least 21) and/or are hot entry level jobs (e.g., first line supervisor is not 
typically filled by someone just entering the industry). Fifteen of the jobs usually require 
a baccalaureate degree. Property/real estate manager, for example, requires a 
baccalaureate degree and the "Occupational Demand and Formal Occupational Supply 
Report" that is included in the State Plan for Vocational and Applied Technology 
Education shows 0 supply anticipated from Secondary Vocational and Applied 
Technology Education (Secondary Voc-Tech). Several of the occupations that are 
described in the Occupational Outlook Handbook as usually requiring or preferring a 
baccalaureate degree are formally e}q)ected to oqperience supply from Secondary Voc- 
Tech. These occupations are noted in the worksheet in Appendix E. 

Thirteen others of the jobs listed in Table 9 usually require some postsecondary 
training. An associate degree is preferred, for example, for entry level positions as 
electrical/electronic technicians. Five of the occupations have formal apprenticeship 
programs. Although 30 to 50 percent of the supply for these occupations is anticipated 
from Secondary Voc-Tech, observers in some parts of the state note that the slots usually 
are filled from within the industry. That leaves an approved list of 22 occupations that a 
young person who is still in or has just graduated from high school could expect to enter 
given appropriate preparation and credentials. It seems reasonable to begin the 
enormous task of developing course specific standards to prepare secondary school 
students to enter and be productive in these 22 occupations. 
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Options for Developing Standards for Courses Linked to Specific Occupations 

Through review of the literature and consultation with oqjcrts in the field of work 
force development, three options were identified as plausible programs for developing 
and implementing course-specific standards to assure that industry standards are met: 
(1) validation of competency profiles that school districts currently are required to 
maintain for each student; (2) use of assessment tools that already are developed and 
being used in other vocational and applied technology education programs across the 
nation; and (3) design, development, validation, implementation, and maintenance of a 
new system. 

Competency Profiles. Section 21.122 of the Texas Education Code says that a 
competency profile must be maintained for each student enrolled in vocational 
education. To facilitate this type of performance evaluation and record keeping, forms 
have been constructed to list the essential elements for specific courses and provide a 
rating scale to show the level of competence achieved by the individual student for each 
of the essential elements of instruction and learning. In one of the school districts that 
was visited during the feasibility assessment, competency profiles are routinely used to 
document the student's progress through a "diploma plan" that is negotiated early in the 
student's tenure in secondary school. In most districts, however, methods for assessing 
competence and actual use of the forms is left to the discretion of the teachers and 
counselors. In many cases, records of course completion serve as documentation of the 
student's occupational competencies. Section 75.211 of the State Board cf Education 
Rules for Curriculum says that "Competency profiles for students in vocational education 
courses, other than grades 11 and 12 occupationally specific courses, shall be defined as 
evidence of mastery of the essential elements."^ ^Nonetheless, one option for 
developing course-specific standards is to review the inventory of competency profiles 
that are being used in school districts, revise them as needed to meet SCANS 
specifications, and conduct an industry validation for each. 

Off-the-Shelf Assessment Tools. At least a dozen associations and independent 
agencies have developed test-item banks and assessment tools to support competency 
based vocational and applied technology education. The Vocational-Technical Education 
Consortium of States (V-TECS) is the most well-known and comprehensive resource for 
industry-validated, competency-based curriculum and assessment materials. The National 
Occupational Competency Testing Institute, Inc. (NOCTI) is a nonprofit corporation that 
develops and administers Student Occupational Competency Achievement Tests 
(SOCAT). A number of state education agencies have undertaken development of 
competency assessment tools. South Carolina, for example, currently is developing 
teacher generated standards and measures with review by industry groups for 90 
occupations. A second option for developing course-specific standards for vocational and 
applied technology education in Texas, therefore, is to identify appropriate ones of these 
already developed tools and link them to specific courses in secondary schools. 
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Development of a New System. The third plausible approach is for the state to 
develop its own course-specific assessment system. This third possibility is suggested in 
written comments submitted to the State Board of Education by staff of the TEA as part 
of the discussion regarding the proposed core standards and measures. The overview 
document stated that additional standards and measures will be developed by "utilizing a 
wide variety of input from personnel throughout the Agency, practicing academic and 
vocational educators, business and industry representatives, and members of the general 
public."22 




Comparing the Options 

Table 10 shows how the different options fared when they were evaluated against 
the seven "benefits" and four "costs" dimensions that were specified in the procedures 
section of this report. The values that are entered in the Table 10 are subjective 
assessments whose basis is e>q)lained in the following paragraphs. In calculating the 
utility value for the different options, equal weight was given to each dimension. That 
means that as a group, the "benefits" dimensions contributed slightly more weight to the 
utility score than did the "costs" dimensions because there are more of them. Inspection 
of Table 10 shows that, at this point in time, the SOCAT, which is an "off-the-shelf 
student assessment system available through NOCTI, has the highest overall utility as a 
strategy for developing course-specific standards for vocational and applied technology 
education in secondary schools in Texas. Noteworthy strengths of the SOCAT system 
are that it is performance based, nationwide, and immediately available. 

Competency Profiles. As mandated by the Rules of Curriculum, competency 
profiles have been constructed to track the student's mastery of essential elements of 
instruction and learning. At the present time there is no assurance that the essential 
elements are themselves industry validated. A major limitation of this option is, 
therefore, that it is based on the traditional approach of "designing up" from the 
curriculum rather than "designing down" from industry standards. The profiles were 
given a score of 0 on this dimension. Because the essential elements applicable across 
all vocational and applied technology education courses include some of the dimensions 
that have been identified as SCANS competencies and foundations-e.g., "demonstrate 
effective communication skills both oral and written and follow through on assigned 
tasks" and "demonstrate productive work habits and attitudes"-the profiles were assigned 
a score of 1 on these two dimensions. As has already been noted, methods for assessing 
student performance usually is at the discretion of the teacher or counselor. The extent 
to which the profiles are performance based is, therefore, unknown at this time. The 
profiles are written to the essential elements of instruction for public schools in Texas 
and do not, therefore, lend themselves to participation in a nation-wide system for 
certifying competencies that students attain in schools. This dimension earned a score of 
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Table 10: Comparison of Options for Course-Specific Standards 







PROFILES 


V-TECS 


SOCAT 


NEW 




BENEFITS DIMENSIONS 










1 


outcome based 


0 


2 


2 


2 


2 


assesses occupational skills 


2 


2 


2 


2 


3 


assesses SCANS competencies 


1 


UK 


UK 


2 


4 


assesses foundation qualities 


1 


UK 


UK 


2 


5 


performance-based 


UK 


-2 


2 


2 


6 


nation-wide 


-2 


2 


2 


-2 


7 


applicable for all students 


1 


UK 


2 


2 
















COSTS DIMENSIONS 










1 


how soon available 


-1 


0 


2 


-2 


2 


administration time 


UK 


2 


1 


2 


3 


development costs 


-1 


0 


0 


-2 


4 


operation/maintenance cost 


-1 


2 


0 


-1 
















AGGREGATE UTILITY SCORE 


0.00 


1.00 


1.44 


0.64 



Scale for Benefits: definitely not (-2) probably not (-1) uncertain (0) probably (1) definitely (2) 
Scale for Costs: very high (-2) high (-1) moderate (0) low(l) very low (2) UnKnown (UK) 



-2. Aitiiough, methods of assessment are discretionary, it seems likely that teachers and 
counselors use the profiles in similar ways with all students, including those who are 
members of special populations. A score of 1 was assigned for this dimension. 

The profiles earned uniform -1 scores on the costs dimensions because, although 
work has already been done to develop the profiles, additional development time and 
dollars will be needed to validate them, cross-walk them to the SCANS competencies 
and foundation qualities, and keep them up to date. The overall utility of this approach 
for developing course specific standards is 0 on the scale that runs from a "worst case" 
whose value would be -2 to "best case" whose value would be +2. 

Off-the-shelf Assessment Tools. Two sources of already developed assessment 
systems were evaluated. A telephone survey of a sample of seven states that have 
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engaged in development of their own assessment system showed the majority of them to 
be linked to V-TECS and/or to have abandoned their development activities. 

Two of V-TECS's products - (1) catalogs that list job duties, speciHc tasks, 
performance steps, and standards for acceptable performance; and (2) criterion- 
referenced test item banks - could be purchased to provide course-specific standards and 
measures of performance for specific courses of study in secondary schools in Texas. 
These materials are outcome based in the sense that they are related to specific 
occupations and are validated by panels of experts from industry. V-TECS earns a score 
of 2 on this dimension. Both the catalogs and the test-item banks are designed to assess 
occupational skills. The system earns a score of 2 for this dimension. At this time the 
extent to which the V-TECS systems assess SCANS competencies and foundations is 
unknown. The test-item banks are limited to paper-and-pencil measures and therefore 
earn a score of -2 for failing to be primarily performance based. The system is, however, 
nation-wide. The ways in which the testing must be adjusted to accommodate students 
who are members of special populations is unknown. 

Although V-TECS has test item banks for a substantial inventory of occupations, 
the overlap with priority occupations in Texas is quite limited. V-TECS's current 
completed and under-development list of test-item banks would address only carpenter's 
helper, maintenance repair worker (small engine repairer), secretary, and several health 
worker specializations under the "other health workers" occupation. The system was 
assigned a score of 0 on the availability dimension because of the wait time that would 
be required to gain access to test-item banks that are addressed to Board approved 
priority occupations. The system is designed to be a useful part of the instructional 
process in the classroom and therefore makes minimal demands for time to administer. 
Although acquisition costs appear to be low at first glance~i.e., catalogs cost $10 to $75 
each and the test item bank costs only $25 on diskette (these are member prices because 
Texas is in the consortium)-a score of 0 was assigned for this dimension because of 
dollar costs that could be incurred in expanding the inventory of assessment tools to 
make a better match with the list of priority occupations in Texas. A score of 2 was 
assigned to the operation/maintenance dimension because Texas would not bear the 
maintenance coste and could disengage from use of the V-TECS at any time. The 
aggregate utility score for taking this approach to developing and implementing course- 
specific standards is 1. 

SOCATs are developed by vocational educators with the cooperation of industry 
personnel. Content is validated by individuals from the world of work and the 
assessment packages are field tested in secondary and postsecondary schools. SOCATs 
are periodically reviewed and revised to ensure that they are current. They earn a score 
of 2 with respect to being outcome based. The tests are designed to assess end-of- 
program accomplishment of occupational skills and earn a score of 2 on that dimension. 
The extent to which the tests assess SCANS competencies and foundation qualities is 
unknown at this time. They use a combination of standardized paper and pencil 
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measures and performance measures. The multiple-choice style paper and pencil test 
can be administered by any teacher or counselor. A journeyman tradesperson or 
business representative with technical expertise in the occupation should administer the 
performance test which, for most occupations, requires approximately 2 to 3 hours. 
Advisory committee members or co-op employers frequently are the ones who administer 
the performance tests for secondary school students. SCXZATs earn a score of 2 for 
being performance-based. The tests are criterion referenced. The information that is 
returned to the student and the school district also provides, however, normative 
information. The student report includes comparisons and a certificate of completion. 
The schoo! receives a class report that provides a composite of the student reports by 
class, school, state, and nation as well as the performance criterion listing for each 
student. SOCATs currently are used statewide in secondary schools in Pennsylvania, 
Tennessee, and Louisiana and are required nationwide for program completers at Job 
Corps centers. The system earns, therefore, a score of 2 for lending itself to the 
purposes of nation-wide evaluation and certification. Each assessment package comes 
with an optional scholastic ability test that encompasses both verbal and nonverbal 
components to facilitate interpretation of the assessment results. This feature makes the 
system applicable for all students, including those who are members of special 
populations. The system earns a score of 2 on this dimension. 

One of the chief advantages of the SOCAT system is that it is available for 
immediate implementation. SOCATs already are available for 14 of the 22 priority 
occupations that young persons can enter while still in or just graduated from high school 
and for 8 of the 13 other approved occupations that do not require apprenticeship or 
baccalaureate degree. SOCAT earns a score of 2 on this dimension. The system does 
impose some time burden for administration-the multiple choice tests usually take 
approximately 3 hours and the performance test 2 to 3 hours. Advanced planning is 
required and NOCTI supplies materials to assist with this planning. Because the tests 
enable the students to demonstrate their skills in context, much of the engaged time can 
be counted as learning time. This dimension has an assigned score of 1. There would 
be no development costs to the state of Texas. The cost for operating the system 
includes a processing fee of $10.50 per student, materials costs for the performance test 
that ranges from $0.00 to $19.92 per student, and costs incurred in bringing the 
journeyman tradesperson or business representative to the test site. These costs are 
largely offset, however, by the fact that updating of the tests is the responsibility of 
NOCTI. The system earned a score of 0 on this dimension. The aggregate utility score 
for this approach for developing course-specific standards for vocational and applied 
technology education in Texas is 1.44. 

Development of a New System. The third option is for the TEA to develop its 
own course-specific assessment system. This approach appears an attractive option 
because the system can be designed to specification. It can be outcome based, primarily 
performance-based, applicable for all students enrolled in vocational and applied 
technology education including those who are members of special populations. It can be 
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designed to assess occupational skills, SCANS competencies, and foundation qualities for 
worlq>lace know-how. By following the guidance of the AEIS and the SCANS, 
developers of the system could minimize the system's demands on instructional and 
learning time. For these reasons, a score of 2 was assigned for all but one of the 
"benefits" dimensions. The one exception was the assigning of a score of -2 because 
development of the state's own system would not lend itself to participation in the 
nation-wide assessment system called for in the America 2000 strategy and recommended 
by the SCANS. Given the current emphasis on redesigning schools and the resources 
that are being made available from SCANS (e.g., task lists constructed around the 
SCANS competencies and foundations for specific occupations) and from other federally 
supported research and development efforts, this option appears very attractive. 

However, there are at least two major obstacles along this path. First, the system 
would not become operational until some time in the future. Although staff at TEA 
have forecast that the system would be developed "within 3 years," this option was 
assigned a score of -2 on this dimension. Research and development activities often take 
longer than is forecast. South Carolina, for example, is developing teacher generated 
standards with review by industry groups for paper-and-pencil tests in 90 occupational 
areas. Because technology is changing so fast, the tests have had to change and are not 
ready at the time when they were expected. Even after the tests are developed, this 
problem will persist - as technology changes in industry, performance assessments must 
also be revised and updated. These costs would show up as maintenance costs that 
would offset any savings realized in operation of an "in-house" system. The second major 
obstacle on this pafh is the very high dollar cost for research, development, and 
validation. Even starting with the advantage of recommended task lists from the 
SCANS, development of paper-and-pencil, portfolio, and/or performance measures is 
expensive. Content validation by panels of oqierts is expensive. Production is ejqsensive. 
These anticipated costs earned a score of -2. Furthermore, the state would have to 
remain committed to its own in-house system-i.e., unlike contracting with an outside 
agency-such as V-TECS or NOCTI-a system purchased by the state's research and 
development dollars could not be easily abandoned even if after a time it appeared to be 
less effective or efficient than another alternative. Maintenance costs, therefore, could 
be very high. A score of -1 was assigned for this dimension. The aggregate utility score 
for developing the state's own assessment system is 0.64. 

The Best Option. Comparison of I 
the aggregate utility scores for the Comparison cf the options showed 

different options showed SOCAT to be SOCAT to offer the greatest benefits at 

the most promising option - i.e., to offer the least costs. 

the greatest benefits for the least costs. It |_ . 

earned 1.44 points on the scale from 

"worst case" value of -2 to "best case" value of +2 compared with 1.00 for V-TECS, 0.64 
for developing a new system, and 0.00 for validating competency profiles that currently 
are used in school districts in Texas. 
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Table 11: Matching SOCATs to Priority Occupations that Do Not 
Require Formal Apprenticeship, Baccalaureate Degree, or Age and Experience 





PRIORITY OCCUPATION SOCAT 

Occupational Skills Training Not Required 

dental assistants dental assisting 

gardeners/groundskeepers 

receptionists 

salespersons, parts 

stock clerks,sales floor 
Occupational Skills Training an Advantage 

billing clerks business data processing 

carpenter's helper carpentry 

diesel mechanics diesel engine mechanics 

electrician's helpers electrical occupations 

legal secretary 

maintenance repairer small engine repair 

mechanic's helper 

medical assistants medical assisting 

medical secretary 

other construction helpers building construction occupations 

other health worker home health aide 

other protective service 

other secretaries general secretarial 

plumber's helpers plumbing 

production clerks production agriculture 

welders & cutters welding 

word processing typists general office 

Postsecondarv Training /College Preferred 

auto mechanics auto mechanics 

child care worker child care services 

computer operators computer & information sciences 

computer programmers/aides computer programming 

drafters drafting 

machinists 

radiologic technicians 

electrical/electronic technicians electronic technology 

food service mgr food production, management, & service 

licensed practical nurse practical nursing 

other engineering technicians 
management support workers 
preschool teachers 
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Cross^lking to Priority Occupations and Specific Courses 

Tlie current inventory of SOCATs includes assessment paclcages for 22 of tlie 35 
Board approved occupations ttiat do not require formal apprenticeship, baccalaureate 
degree, or minimum age and experience. The cross-walk of SOCAT to the list of 
approved occupations is shown in Table 11. Fourteen of these matches were for 
occupations that a young person could expect to enter while still in or just graduated 
from high school-i.e., those that do not prefer or require postsecondary training, college, 
or minimum age or experience. Eight additional matches were made for occupations in 
which postsecondary training or college is preferred but baccalaureate or formal 
apprenticeship is not preferred or required. 

The SOCAT for each of the 22 occupations matched as shown in Table 11 also 
was cross-walked to specific courses in vocational and applied technology education in 
secondary schools in Texas. This task was accomplished by matching essential elements 
in the Rules for Curriculum with elements listed as areas of assessment in the SOCAT. 
Results of this cross-walking showed that there were no occupation-SOCAT matches in 
marketing education. There was 1 in agricultural education, 4 in health occupations, 2 in 
home economics, 3 in office education, 2 in industrial technology, and 10 in trade and 
industrial education. Table 12 shows results of the cross-walk to the curriculum for one 
course in each of the program areas. The detailed analysis for all 22 occupation-SOCAT 
matches is in Appendix F. 

With the exception of production clerk/production agriculture, which was the only 
occupation-SOCAT match related to agricultural education, each of the SOCATs and 
matching courses that was selected for display in Table 12 was chosen because tJjere 
already is an industry or professional certification ladder for the specific occupation. 
For secref jries, testing and certification of entry-level skills is available through the 
Office of Proficiency Assessment and Certification (OPAC) program of Professional 
Secretaries International (PSI). Advanced certifications can be earned by passing a 
series of exams offered by PSPs Institute for Certifying Secretaries. Carpenter's Helpers 
can seek to enter forma! apprenticeship programs to obtain a series of entry level, 
intermediate, and journeyman certifications as a Carpenter through the local chapters of 
carpenters' and contractors' unions. Medical Assistants obtain the Certified Medical 
Assistant Credential through the American Association of Medical Assistants. For child 
care workers the Child Development Associate (CDA) credential is offered by the 
Council for Early Childhood Professional Recognition. Award of the CDA is based on a 
formal skills assessment. Computer Operators can be certified through the Institute for 
Certification of Computer Professionals by passing a core exam and exams in two 
specialty areas. 
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Table 12. Matching SOCATs to Specific Courses of Study 



occupation - SOCAT - program area 



Specific Courses of Study 



other secretaries - general secretarial 
- Office Education 



production clerk - production agriculture 
- Agricultural Education 



carpenter's helper - carpentry 
- Trade and Industrial Education 



Office Administration 
Business Information Processing 
Business Office Services 
Business Computer Applications 
Office Support Systems 
Microcomputer Applications 

Agribusiness Management & Marketing 

Animal Science 

Plant and Soil Science 

Introduction to Agricultural Mechanics 

Building Trades 
Construction Carpentry 
Introduction to Construction Careers 




medical assistants - medical assistant 
- Health Occupations Education 



Health Care Science 
Electrocardiograph Technician 
Medical Lab Aide 
Nursing Assistant, Radiology 
Mtukcal Office Assistant 



child care worker - child care services 

- Home Economics Education 

computer operator - computer and 
information sciences 

- Industrial Technology Education 



Child Care and Guidance, Management 
and Services 

Business Information Processing 
Business Computer Programming I 
Business Computer Applications I 



NOTE: The analysis indicates that the student would need to take all of the courses 
listed in the block to achieve mastery of the specific knowledge and skill areas assessed 
in the relevant SOCAT- 




Four of the occupations - secretary, carpenter's helper, medical assistant, and 
computer operator - are jobs for which the SCANS has developeu task lists linked to the 
SC/^S competencies and foundation 
skills and qualities (see Appendix C of 
this document for more detailed 
information about the SCANS task lists 
for these priority occupations for Texas). 
These four occupations - which have (1) 
an existing industry-based assessment and 
certification system, (2) an up-do-date 
task analysis linked to the SCANS 
competencies and foundations; (3) the 
education-based, nation-wide assessment 
system offered in the SOCATs, and (4) an 
existing block of vocational and applied 
technology education courses to enable the student to achieve the necessary skills - are 
especially attractive as candidates for pilot testing of the utility of the SOCATs. 

If the SOCATs for these four occupations were adopted statewide for a 
longitudinal study of the utility of using the SOCATs as one element of the emerging 
nation-wide system for assessing and certifying levels of proficiency attained in school 
and on-the-job, the projected cost of administering the SOCATs would be less than 
$30,000 per year. The cost shown in Table 13 for each SOCAT includes the $10.50 per 
student processing fee and the per student cost of materials consumed in conducting the 
performance assessment. The projected total cost is based on the assumption that every 
secondary voc-tech student that is forecast as contributing to the total supply for the 
occupation would take the SOCAT.^^ Table 13 also shows the percent of total supply 
that is forecast as coming from secondary vocational and applied technology education. 



Table 13: Projected Costs for Administering these Four SOCATs Statewide 


SOCAT 


COST PER 


STUDENTS 


% OF SUPPLY 


TOTAL 


carpentry 


$10^0 


950 


64 


$ 9,738 


medical assisting 


$10^0 


99 


2 


$ 1,040 


computer operator 


$12^0 


1,164 


13 


$ 14,550 


general secretarial 


$11.50 


161 


41 


$ 1,852 










$ 27,179 
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Secretary, carpenters helper, medical 
assistant, and computer operator are 
especially attractive candidates for pilot 
testing the udlity of SOCAT as an 
assessment system to provide course- 
specific standards for vocadonal and 
applied technology education in 
secondary schools in Texas. 



If all 22 of the SOCATs that match Board approved priority occupations that do 
not require formal apprenticeship, baccalaureate degree, or age and experience were 
adopted state-wide ami every secondary vocational and applied technology education 
student that is forecast as contributing to the supply for the related occupations were to 
take the tests, the total projected cost per year for administering the SOCATs in Texas 
would be less than $755,000. For fiscal year 1992, this amount of money represents 
approximately 1 percent of the total federal grant to Texas under the Perkins Act and 
less than 10 percent of the federal grant monies that are projected to be spent on state 
administration and leadership for Vocational and Applied Technology Education. 
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Section 5: EVALUATION PROCEDURES AND FINDINGS 

The project description for 1991-1992 indicates tliat regular and frequent 
formative evaluations would be conducted. The methods specified for these evaluations 
were (1) providing opportunity for input, criticism, and evaluation by members of the 
general advisory committee and course-specific advisory groups; (2) in-house evaluation 
using the expertise of personnel at the EIMC; and (3) bimonthly and quarterly 
administrative reports to the Director of Vocational Education Programs at the TEA. 

Input and Evaluation by Advisory Committees. In August 1991, a series of nine 
teleconferences were conducted with nine Course-Specific Advisory Groups. Tape 
recordings of these sessions and notes taken by staff at the EIMC during the conferences 
were used to guide revisions of the drafts of the Core Performance Standards prior to 
the October meeting of the General Advisory Committee. During project year 1991- 
1992, the General Advisory Committee met once in October 1991 to critique content and 
format of the core performance standards and the first draft of the prototype data 
collection instrument. The minutes of that meeting show that the committee provided 
direction for revising and finalizing the set of standards that were taken to school 
districts for feasibility assessment in November and December of 1991 (see Appendix B). 

In-house Evaluations at EIMC. Review of progress on this project by staff of the 
EIMC at the end of August 1991 revealed that significant delays in the work had been 
created by the illness of the project manager. Problems caused by these delays were 
compounded when staff at the TEA determined that the due date for delivery of 
feasibility-tested core performance standards should be moved from April to January 
1992 so as to ensure timely implementation of the evaluation system required by the 
Perkins Act of 1990. Staff at the EIMC, with advice and consent of TEA's directors of 
Vocational Education Programs and Vocational Special Needs, accepted the resignation 
of the project manager in September and began a search for a new project manager. In 
early October 1991, the new project management team met twice with content area 
specialists at the EIMC to review the history of the project and obtain guidance from 
these specialists regarding status of the core performance standards, strategies for 
conducting the feasibility assessment, and recommendations regarding course-specific 
standards. Throughout the period November 1991 - June 1992, individual members of 
the EIMC were consulted for their evaluation and guidance regarding the development 
of core and course-specific standards and measures of performance. 

Progress Reports to the TEA. The written quarterly progress reports are in 
Appendix G. These reports include discussion of guidance received in monthly meetings 
with the project director and quarterly meetings with the group of vocational and applied 
technology education program directors at the TEA. In addition to the written reports 
and quarterly meetings, project staff participated with the Director of Vocational 
Education Programs and other personnel at the TEA in meetings at least once a month 
to review progress and guide the project. 
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Section 6: CONCLUSIONS, IMPLICATIONS, AND RECOMMENDATIONS 

Condusions and Recommendations Regarding Core Standards 

The three standards approved by the Board for school year 1992-1993 have some 
strengths and some vulnerabilities* One of the strengths is that the standards are in 
compliance with the Section 115 of the Perkins Act which says that the system must 
include a measuring of learning and competency gains, including student progress in the 
achievement of basic and more advanced 
academic skills and at least one other 
measure of competency attainment, job or 
work skill attainment or enhancement, 
retention in or completion of secondary 
school or its equivalent, and/or placement 
into additional training or education, 
military service, or employment. Standard 
number 1, which says that at least 90 
percent of students who were enrolled in 
a coherent sequence of courses 
invocational and applied technology 
education will pass the TAAS exit test, 
specifies a measure of achievement of 
basic academic skills. Standard number 2, 
which says that at least 95 percent of 12th 
grade students who were enrolled in a 
coherent sequence of courses in 
vocational and applied technology 
education programs will demonstrate 
competency in occupational skills 
constitutes a standard and measure of 
performance relative to job or work skill 
attainment. Standard number 3 sets a 

standard and specifies measurement of placement into additional training or education 
(i.e., enrolled in a postsecondary educational institution or enrolled in a registered 
apprenticeship program or in training related to their vocational education program), 
military service, or employment (i.e., employed in a paid or unpaid job related to their 
vocational and applied technology education and training). 

A second strength of the set of three standards and related measures of 
performance is that they will serve one of the purposes of the required annual evaluation 
as specified in Section 117 of the Act. They will help school districts evaluate progress 
in providing students with strong experience in and understanding of the industry they 
are preparing to enter. 



Strengths of the core standards approved 
by the State Board of Education for school 
year 1992-1993 are that they: 

* meet the requirements cf Section 
115 of the Perkins Act which says 
that the system must incbide a 
measure of achievement cf 
academic sldUs and at least one 
other measure of competency 
attainment; and 

* will serve the purpose specified in 
Section 117 of helping school 
districts evaluate progress in 
providing students with experience 
in and understaruiing of the 
industries they are preparing to 
enter. 
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These standards fall short, 
however, of being useful for two other 
purposes that are mandated in Section 
117. As currently stated, implementation 
of the core standards and measures of 
performance will not help school districts 
to (A) identify and adopt strategies to 
overcome barriers for individuals who are 
members of special populations or (B) 
evaluate progress of individuals who are members of special populations. 

The standards also fall short of helping school districts comply with data gathering 
and reporting requirements that are not directly connected to the evaluation 
requirements but that could be advantageously addressed in the statewide system of 
standards and measures. As was discussed in the introduction to this report. Section 111 
of the Act requires, for example, that data be gathered, analyzed, and disseminated on 
the adequacy and effectiveness of vocational and applied technology education programs 
in meeting education and employment needs of women, including those who are 
displaced homemakers and single parents. As currently configured, the standards and 
measures of performance approved by the State Board of Education for school year 
1992-1993 do not provide any direction or assistance with these kinds of related data 
gathering and reporting requirements. 

These observations endorse the conclusion drawn by staff of the TEA when they 
presented the three standards and related measures of performance to the Board that 
"other indicators of program effectiveness must be developed."^ ^ One approach that 
can be taken toward developing these additional indicators is to review the work 
accomplished by the General Advisory Committee in developing standards for evaluating 
outcomes for students who are members of special populations. Number 6 of the core 
standards and measures of performance that were recommended to the TEA based on 
the Committee's work and results of the feasibility assessment conducted as part of tliis 
project states that members of special populations should be enrolled in vocational and 
applied technology education courses at rates that are consistent with percentages of 
those groups of students within the school district. This standard is directly addressed to 
the issue of access. Number 7 of the standards and measures recommended to the TEA 
in January of 1992 is addressed to success in vocational and applied technology 
education programs attained by those who are members of special populations. 

If these two standards addressed to access and success for special and targeted 
populations were to be added to the approved list, the related data collection activity 
would help the districts to capture other information that the Perkins Act demands. As 
was noted in the introduction to this report. Section 111 of the Act requires that states 
(and, therefore, districts) identify the numbers of women and men, students with 
handicaps, students with economic disadvantage, and students with limited English 
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As currently stated, the core standards fall 
short of being useful in helping to focus 
Federal funds on inyjroving vocadonal 
education and, in particular, on improving 
vocational education and services for 
members cf special populations. 
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proficiency and ensure that needs of such students for participation in vocational and 
applied technology education programs are being met. Data gathering and conriputations 
to compare outcomes against standard for proposals 6 and 7 (see Table 6 in this report), 
would make this information readily available. For the present, it is recommended that 
school districts be encouraged to use the data gathering instrument that is included in 
Appendix D. This instrument, which addresses the three standards and measures 
approved by the Board gives districts the option of compiling information for targeted 
groups and special populations and then aggregating results to compare overall 
performance against the standards. 

Another area of concern regarding core standards and measures of performance is 
that no strategy has yet been proposed for integrating this federally required evaluation 
into the Academic Excellence Indicator 
System that is being developed by the 
TEA in response to sections 21.753 and 
21.7531 of the Texas Education Code. 
State law requires that the set of criteria 
for evaluating performance of school 
districts must address the quality and 
effectiveness of the districts' vocational 
education programs. At the present time 
the strategies that have been proposed for 
meeting this requirement are to measure 
postsecondary performance in the 
workplace and/or to monitor percent of 
students completing approved 
"preparatory programs" which may or may 
not include "voc/tech-prep" as an option. 
It seems wise, therefore, to conclude that particular attention should be given to 
determining how the annual evaluation of vocational and applied technology programs 
that school districts must conduct in order to comply with terms of the Perkins Act can 
be used to meet the state mandate that performance of the districts' vocational education 
programs be addressed in the set of criteria for district evaluation, accreditation, and 
determination of exemplary status. 

Conclusions and Recommendations Regarding Course-Specific Standards and Measures 
of Performance 

In making their recommendations to the State Board of Education regarding core 
standards and measures of performance for vocational and applied technology education, 
staff of the TEA indicated that "appropriate performance teste must be developed and 
validated, to ensure that industry standards are being met."^°The Student Occupational 
Competency Achievement Teste (SOCAT) already developed, validated, and periodically 
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At the present time there is no link 
between strate^esfor meeting 
requirements cf the Perkins Act and 
proposals for meeting requirements cf the 
Texas Education Code that criteria for 
evaluating performance of school districts 
must address the quality and 
effectiveness of the districts' vocational 
education programs •• i.e., the core 
standards are not integrated into the 
AEIS. 
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updated by the National Occupational Competency Testing Institute (NOCTI) offer a 
very positive alternative to the costly and time-intensive business of developing standards 
from scratch for programs in Texas. 

The SCXZATs are industry 
validated, field tested in school districts, 
performance-based, and criterion- 
referenced. Because they are in use 
locally in countless school districts, state- 
wide in at least 3 states, and nationwide 
in the Job Corps Centers, normative 
references also are produced in the 
reports that school districts and students 
receive when they participate in SOCAT. 
At the present time, SOCATs are 
available to match 14 of the 22 Board 
approved priority or emerging occupations 
that a young person could expect to enter wi\ile still in or just graduated from high 
school and for 8 of the 13 approved occupations that require postsecondary training but 
do not require formal apprenticeship or baccalaureate degree. It is therefore 
recommended that a longitudinal study be initiated in the fall of 1992 to conduct an 
empirical evaluation of the utility of the SOCAT as a strategy for linking industry 
standards, certification of workplace competencies, and secondary school-based 
vocational and applied technology education. 

Analysis of the match between available SOCATs, the list of Board approved 
occupations, and specific courses of study in vocational and applied technology education 
in Texas indicates that Carpenter's 
Helper, Computer Operator, Medical 
Assistant, and Secretary ai e especially 
attractive as occupations with which to 
begin the development of appropriate 
performance tests to ensure that industry 
standards are being met. For each of 
these occupations there is at least one 
applicable SOCAT and at least one block 
of two to four vocational and applied technology education courses whose essential 
elements would prepare the student to demonstrate achievement of necessary skills for 
being work ready. 

These occupations also are among those for which the SCANS has published a 
task list that is cross-walked to the SCANS competencies and foundation skills and 
qualities. In addition, they are among the 700 occupations for which competency listings, 
personal skills inventories, and personal interest inventories are maintained in system 
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The Student Occupational Competency 
Achievement Tests (SOCAT) that 
already are used statewide in three states 
and nationwide in Jobs Corps Centers 
hold substantial promise as a system for 
Texas to use to link industry standards, 
certification cf workplace competencies, 
and specific courses cf study in 
vocational and applied technology 
education. 



For more information regarding... 

SOCAT (800) 334-6283 

Quality Wo± Fone Planning . . . (512) 475-3428 
SOICC (512)463-2399 





used for Quality Work Force Planning in Texas - i.e., in the data base of the State 
Occupational Informational Coordinating Committee (SOICC). Each of these four entry 
level occupations has access to a progressive skills certification system that is industry- or 
association-based. 

Two of these occupations - medical assistant and secretary - have career 
advancement opportunities based on experience in the profession, one - carpenter's 
helper - can progress into a formal apprenticeship, and one - computer operator - 
requires postsecondary training or college education. These occupations therefore are 
especially attractive as test cases for development and validation of a skills training and 
certification system that will meet industry needs for a skilled work force and broaden 
young people's options for making the transition from school to progressively more 
challenging and rewarding work. Because the SOCATs already are available and 
because the projected cost for administering them for just these four occupations for 
secondary vocational and applied technology education programs in Texas is small-i.e., 
less than $30,000 per year-it is recommended that a pilot study be undertaken to 
evaluate the utility of the SOCATs and to explore this option for linking industry 
standards, certification of workplace competencies, and specific courses of study in 
vocational and applied technology education in secondary schools in Texas. 

These conclusions and recommendations regarding course-specific standards are 
summarized together with those regarding core standards and measures of performance 
in Table 14. As is noted in Table 14, the specific recommendations that emerge from 
this study are: 

1) Additional indicators of program effectiveness must be developed to meet the 
requirements of the Perkins Act of 1990, giving particular attention to indicators 
that will ensure adequate evaluation and feedback to ensure access and success in 
vocational and applied technology education programs for individuals who are 
members of special populations; 

2) In the interim, school districts can use the prototype data collection instrument 
that is included in appendix D to collect, analyze, and use data to evaluate 
program performance relative to the three core standards approved for school 
year 1992-1993 and address data gathering and evaluation requirements regarding 
program reach and effectiveness for members of targeted groups and special 
populations; and 

3) A pilot study should be conducted to evaluate the utility of the SOCAT as a 
system of course-specific standards and performance measures to ensure that 
industry standards are met. 
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Table 14 Overview of Conclusions and Recommendations 



R^arding Core Standards and Measures of Performance.^ 

* In addition to the three standards approved by the Committee of Practitioners 
and the State Board of Education for school year 1992-1993, other indicators 
relating to program effectiveness must be developed. 

Standards approved for 1992-1993 do not directly serve all of the purposes 
identified in the Perkins Act of 1990 - e.g., the standards will not be 
directly useful in helping school districts to identify and adopt strategies to 
overcome barriers to access and success for individuals who are members 
of special populations as is mandated in Section 117 of the Act 

The standards also fall short of helping school districts comply with data 
gathering and reporting requirements that are not directly connected to the 
evaluation requirements but could be advantageously addressed in the 
statewide system of standards and measures - e.g., section 111 of the Act 
requires that data be gathered, analyzed, and disseminated on the 
adequacy and effectiveness of vocational education programs in meeting 
education and employment needs of women. 

* Strategies for integrating the evaluation that is required by the Perkins Act into 
TEA'S Academic Excellence Indicator System need to be developed in order to 
meet the State's requirement under section 21.753 of the Texas Education Code 
quality and effectiveness of the districts' vocational education programs must be 
addressed in criteria for evaluating performance of school districts. 



R^arding Course-Specific Standards and Measures of Performance.- 

* The Student Occupational Competency Achievement Tests (SOCAT) that already 
are used statewide in 3 states and nationwide in Jobs Corps Centers hold 
substantial promise as a system for Texas to use to link industry standards, 
certification of workplace competencies, and specific courses of study. 

* Carpenter's Helpei, Computer Operator, Medical Assistant, and Secretary are 
Board approved priority occupations that are especially attractive for evaluating 
the utility of the SOCATs for meeting federal and state assessment requirements 
and for broadening students' options for transition from school to progressively 
more challenging and more rewarding opportunities in the work force. 
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PROJECT DESCRIPTION ABSTRACT 
1991-92 



Objectives 

This project will have two main objectives. This first will be to continue the development of 
performance standards for sequences of vocational courses. The second will be to conduct a feasibility 
study of the core performance standards to ensure that those standards are clearly defined, cost 
efficient, manageable in number, relatively easy to measure, timely, goal oriented, attainable yet 
credible, and focused on product rather than process. 



Procedures 

A general advisory committee will review and approve the conduct of both major portions of this 
project. This project will examine courses other than those considered during the 1990-91 project and 
determine for which courses, if any, additional performance standards need to be developed. A 
feasibility study will be conducted to determine whether the core standards developed during the 1990- 
91 project are realistic, and adjustments will be made, if necessary; this study will greatly enhance the 
process of adopting and implementing the standards. Course-specific performance standards will assist 
local administrators in the evaluation of their programs. All standards will be clearly defined, cost 
efficient, manageable in number, relatively easy to measure, timely, goal oriented, attainable yet 
credible, and focused on product rather than process. 

Product(s) to Be Delivered 

Three final products will be developed: (1) course-specific performance standards for those vocational 
courses for which they are needed; (2) a feasibility study of the core performance standards developed 
during the 1990-91 school year; and (3) a data-gathering instrument to be used in the feasibility study. 
The instrument could be used by vocational administrators in local school districts in data collection for 
their districts' evaluation required under the performance standards. One (1) camera-ready copy and 
100 additional copies of the performance standards, the data-gathering instrument, and the results of 
the feasibility study will be produced. 



Quality Control ofProduct(s) 

Regular and frequent formative evaluations will be conducted during the development of the 
performance standards and the feasiblity study. Advisory and validation committees— composed of 
individuals with experience and expertise appropriate to this specific project— will be formed. A 
general advisory committee and numerous course-specific advisory groups will provide ample 
opportunity for input, criticism, and evaluation. This process will also improve acceptance of the final 
product by teachers and administrators. Regular and comprehensive in-house evaluation will take 
place during the course of the project using the expertise of EIMC. Bimonthly reports with copies of the 
products being developed, quarterly adminstrative reports, and a comprehensive final report will be 
submitted to the Director of Vocational Education Programs at the Texas Education Agency. 
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PERPORMMICE SIftNCftRDS FOR 
VOCATIONAL AND APPLIED TEXSNOLDGY EDQCATICaJ 

First Advisory Onmnittee liBet±ng 
Oc±cber 21, 1991 





Ihe first advisory ccanmittee meeting was held an Kbrxiay, October 21st 
frotti 10:00 a.m* to 2:00 p.m. at the Hionpson Conference C3enter on the 
University of Texas Caitpus. A list of the Meatibers in attendance is 
attached. 

Dr. BariDara Selke-Kern, Director of Materials Develc^ment, Extension 
Instruction and Materials Center (EIMC) , c^jened the meeting with welocming 
remarks. She informed the cornniittee of the change in Project Managers. 
Richard Bouton, unfortunately, is ej5)eriencing health prxi^bleros vdiich has 
prohibited him from cotpleting this project. She introduced the new 
Project Manager, Dr. Cin*^ Roberts-Gray. Dr. Cixtiy Roberts-Gray gave a 
brief ijpdate on the project and dlyyaissed her collaboration with Ridiard. 
She had met with him the week prior to ensure that his previous work and 
ideas were incorporated into her subsequent work on the project. 

' Dr. Roberts-Gray presented the Core Performance Standards which needed 
to be finalized by the close of the afternoon meeting. Ccnpletion of the 
standards was inportant because they will be tested in visits to school 
district viiich will take place in November and early December. She 
ejq^lained that the purpose of the feasibility stud/ was to determine if the 
criteria in the standards were reasonable and realistic and if the 
information requested could be ctotained frcsn the districts. The final Core 
Standards will be produced in mid-January and the Course-Specific Standards 
would be finalized in the spring of 1992. Dr. Rcberts-Gray believes that 
these will take the form of amendments and addenda to the Core Standards. 

Discussion and revision of each of the eleven standards followed. 

Standard 1 : The question arose as to vdiat constitutes a "coherent 
sequerK::e" and vdiether this sequence should be made xsp of courses across 
programs or only program specific courses. 

Sylvia Clark, Texas Education Agency, believes that a sequence of broad to 
specific courses constitutes a sequence and that one already exists. 

Barbara Terrell, Director of Programs, Health Ocojpations Education, felt 
that there is no coherent sequence of courses. 

RciDert Patterson, Director of Vocational Programs from the Texas Education 
Agency, reminded the Members that the law and the annual application 
requires a coherent sequence of courses. Mr. Patterson suggested that a 
sequence of courses should be at least 4 credits. He eLLso suggested that 
districts should begin to offer 1 sequence and expand. Many small scdiools 
do not have these in place but should begin to work tcwards developing 
them. Due to these new requirements a longitudinal study of the 
iitprovements should be considered. 
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A discussion on the necessity of the use of the TMS test to indicate 
ccnr^tency attainment vas initiated by Will Reese, Texas Council on 
Vocational Education. It was suggested that a pre-, post-test be 
consider^ or another academic indicator be used. 

It was decided that Standard 1 would be labeled under developnent mtil the 
feasibility study was conducted. 

Stanf^^rd 2 ; The tiioing of the evaluations was brou^^t into discussion 
and it was determined that they will be conducted at the end of each 
academic year. 

Agreement was reached that the reqaireroent of 4.5 state-approved units of 
credit to be obtained per year by students enrolled in two semesters of 
vocational education courses was insufficient. The concensus was to change 
4.5 to 5.5 units of state approved required and/or elective units of 
credit. 

SancSrea MarveJL, retired Vocational Director, Wichita Falls Independent 
School District, recoramended that the evaluation to be attentive to whether 
districts vAiich have 7 period days provide a loophole for the 5.5 unit 
credit requirements. 

Standard 3 and 4 : Few wording changes were suggested. 

Standard 5 and 6; Few */ording changes were suggested. Dr. Roberts-Gray 
suggested the possibility of reviewing attendenoe records as a method for 
collecting data on these Standards. 

Standard 8 : The decision was reached to raodify this standard cotpletely. 
The revised version of this and all other standards are inclxjded in the 
attachments. 

Standard 9. 10. and 11 : Few wording changes were suggested. 



After all Standards were discussed. Dr. Roberts-Gray told Members that 
the ammended version of the Core Standards would be mailed within two days 
of the meeting for their approval and conments. 

The meeting was adjomed at 2:00 p.m. 
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Skills and Tasks for Jobs 
Cross-walk to the SCANS Competencies and Foundation 
"Most Critical" skills and tasks are listed by their rank ordering 

taken from 

Skills and Tasks for Jobs: A SCANS Report for AMERICA 2000 
U.S. Department of Labor, February 1992 



OCCUPATION COMPETENCIES FOUNDATION 

Carpenter C18, C09, C15, C19, CIO, F13, F03, F05, F04, FOl, 

C07 F17 



Computer Operator Cll, COl, C09, C15, C20, F13, FOl, F02, F09, F17, 

C19, C16, CIO, COS F05, F06, Fll 



Medical Assistant C09, COl, Cll, C06, COS, F17, F13, F14, F15, F05, 

CIS, C19, C07 Fll, FOl, F16 



Secretary C09, COl, Cll, COS, COS, FOS, F02, F06, FOl, F13, 

C14, CIS, C06, C07, C19 FOS 



The SCANS report includes the task analysis for a total of SO jobs. 
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Table Definitions of Competencies and Foundation Skills 



Competencies 

Resources 




CI Allocates Time • Selects relevant, goal-related activities, ranks them in order of 
importance, allocates time to activities, and understands, prepares, and follows 
schedules. Competent performance in allocating time includes properly identifying 
tasks to be completed; ranking tasks in order of importance; developing and 
foDowing an effective, workable schedule based on accurate estimates of such 
things as importance of tasks, time to complete tasks, time available for 
completion, and task deadlines; avoiding wasting time; and accurately evaluatmg 
and adjusting a schedule. 

C2 Allocates Money • Uses or prepares budgets, including making cost and revenue 
forecasts, keeps detailed records to track budget performance, and makes 
appropriate adjustments. Competent performance in allocating money includes 
accurately preparing and using a budget according to a consistent and orderly 
accounting method; accurately calculating future budgetary needs based on 
projected costs and revenues; accurately tracking the extent to which actual costs 
and revenues differ from the estimated budget, and taking appropriate and 
effective actions. 

C3 Allocates Material and Facility Resources • Acquires, stores, and distributes 

materials, supplies, parts, equipment, space, or final products in order to make the 
best use of them. Competent performance in allocating material and facility 
resources includes carefully planning the steps invohred in the acquisition, storage, 
and distribution of resources; safely and efficiently acquiring, transporting or 
storing them; maintaining them in good condition; and distributing them to the 
end iiser. 



C4 Allocates Human Resources - Assesses knowledge and skills and distributes work 
accordingly, evaluates performance and provides feedback. Competent 
performance in allocating human resources includes accurately assessing people's 
knowledge, skills, abilities, and potential; identifying present and future workload; 
making effecthre matches between individual talents and workload; and actively 
monitoring performance and providing feedback. 



Information 




C5 Acquires and Evaluates Information - Identifies need for data, obtains them from 
existing sources or creates them, and evaliiates their relevance and accuracy. 
Competently performing the tasks of acquiring data and evaluating information 

2-4 
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includes posing analytic questions to determine specific information needs; 
selecting possible information and evaluating its appropriateness; and determming 
when new information must be created. 

C6 Organizes and Maintains Information - Organizes, processes, and maintains 

written or computerized records and other forms of information in a systematic 
fashion. Competently performing the tasks of organizing and maintaining 
information includes understanding and organizing information from computer, 
visual, oral and physical sources in readUy accessible formats, such as 
computerized data bases, spreadsheets, microfiche, video disks, paper files, etc.; 
when necessary, transforming data into different formats in order to organize them 
by the application of various methods such as sorting, cl issifying, or more formal 
methods. 

C7 Interprets and Communicates Information - Selects and analyzes information and 
communicates the results to others using oral, written, graphic, pictorial, or multi- 
media methods. Competently performing the tasks of communicating and 
interpreting information to others includes determining information to be 
communicated; identifying the best methods to present information (e.g., 
overheads, handouts); if necessary, converting to desired format and conveying 
information to others through a variety of means including oral presentation, 
written communication, etc. 

C8 Uses Computers to Process Information - Employs computers to acquire, 

organize, analyze, and communicate information. Competently using computers to 
process information includes entering, modifying, retrieving, storing, and verifying 
data and other information; choosing format for display (e.g., line graphs, bar 
graphs, tables, pie charts, narrative); and ensuring the accurate conversion of 
information into the chosen format. 





Interpersonal 

C9 Participates as a Member of a Team - Works cooperatively with others and 
contributes to group with ideas, suggestions, and effort. Demonstrating 
competence in participating as a member of a team includes doing own share of 
tasks necessary to complete a project; encouraging team members by listening and 
responding appropriately to their contributions; building on individual team 
members' strengths; resohdng differences for the benefit of the team; taking 
personal responsibUity for accomplishing goals; and responsibly challenging existing 
procedures, policies, or authorities. 

CIO Teaches Others - Helps others learn. Demonstrating competence in teaching 
others includes helping others to apply related concepts and theories to tasks 
through coaching or other means; identifying training needs; conveying job 
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information to allow others to see its applicabUity and relevance to tasks; and 
assessing performance and providing constructive feedback/reinforcement. 

Cii Serves Qients/Customers - Works and communicates with clients and customers to 
satisfy their expectations. Demonstrating competence in serving clients and 
customers includes actively listening to customers to avoid misunderstandings and 
identifying needs; communicating in a positive manner especially when handling 
complaints or conflict; efficiently obtaining additional resources to satisfy client 
needs. 

C12 Ryercises Leadership - Communicates thoughts, feelings, and ideas to justify a 

position, encourages, persuades, convinces, or otherwise motivates an individual or 
groups, including responsibly challenging existing procedures, policies, or authority. 
Demonstrating competence in exercising leadership includes making positive use 
of the rulesA'alues followed by others; justifying a position logically and 
appropriately; establishing credibility through competence and integrity; and taking 
minority viewpoints into consideration. 

C13 Negotiates to Arrive at a Decision - Works toward an agreement that may involve 
exchanging specijQc resources or resohdng divergent interests. Demonstrating 
competence in negotiating to arrive at a decision involves researching opposition 
and the history of the conflict; setting realistic and attainable goals; presenting 
facts and arguments; listening to and reflecting on what has been said; clarifying 
problems and resolving conflicts; adjusting quickly to new facts/ideas; proposing 
and examining possible options; and making reasonable compromises. 

C14 Works with Cultural Diversity - Works well with men and women and with a 

variety of ethnic, social, or educational backgrounds. Demonstrating competence 
in working with cultural diversity involves understanding one's own culture and 
those of others and how they differ, respecting the rights of others while helping 
them make cultural adjustments where necessary; basing impressions on individual 
performance, not on stereotypes; and understanding concerns of members of other 
ethnic and gender groups. 



Systems 

C15 Understands Svstems - Knows how social, organizational, and technological 

systems work and operates effectively within them. Demonstrating competence in 
understanding systems involves knowing how a system's structures relate to goals; 
responding to the demands of the system/organization; knowing the right people 
to ask for information and where to get resources; and functioning within the 
formal and informal codes of the social/organizational system. 




Monitors and Corrects Performance • Distinguishes trends, predicts impact of 
actions on system operations, diagnoses deviations in the function of a 
system/organization, and takes necessary action to correct performance. 
Demonstrating competence in monitoring and correcting perforr-ince includes 
identifying trends and gathering needed information about how the system is 
intended to function; detecting deviations from system's intended purpose; 
troubleshooting the system; and making changes to the system to rectify system 
functioning and to ensure quality of product. 

C17 Improves and Designs Systems - Makes suggestions to modify existing systems to 
improve products or services and develops new or alternative systems. 
Demonstrating competence in improving or designing systems involves making 
suggestions for improving the functioning of the system/organization; 
recommending alternative system designs based on relevant feedback; and 
responsibly challenging the status quo to benefit the larger system. 





Technology 

C18 Selects Technology - Judges which set of procedures, tools, or machines, including 
computers and their programs, will produce the desired results. Demonstrating 
competence in selecting technology includes determining desired outcomes and 
applicable constraints; visualizing the necessary methods and applicable 
technology; evaluating specifications; and judging which machine or tool will 
produce the desired results. 

C19 A pplies Technolo^ to Task - Understands the overall intent and the proper 

procedures for setting up and operating machines, including computers and their 
programming ^tems. Demonstrating competence in how to apply technology to 
task includes understanding how different parts of machines interact and how 
machines interact with broader production systems; on occasion installing 
machines including computers; setting up machines or systems of machines 
efficiently to get desired results; accurately interpreting machine output; and 
detecting errors from program output. 

C20 Maintains and Troubleshoots Technology - Prevents, identifies, or solves problems 
in machines, computers, and other technologies. Demonstrating competence in 
maintaining and troubleshooting technology includes identifying, understanding, 
and performing routine preventative maintenance and service on technology; 
detecting more serious problems; generating workable solutions to correct 
deviations; and recognizing when to get additional help. 




Foundation Skills 



Basic Skills 

Fl Reading • Ix)cates, understands, and interprets written information in prose and 
documents— including manuals, graphs, and schedules— to perform tasks; learns 
from text by determining the main idea or essential message; identifies relevant 
details, facts, and specifications; infers or locates the meaning of unknown or 
technical vocabulaiy; and judges the accuracy, appropriateness, style, and 
plausibility of reports, proposals, or theories of other writers. 

F2 Writing - Communicates thoughts, ideas, information, and messages in writing; 
records information completely and accurately; composes and creates documents 
such as letters, directions, manuals, reports, proposals, graphs, flow-charts; uses 
language, style, organization, and format appropriate to the subject matter, 
purpose, and audience; includes supporting documentation and attends to level of 
detail; and checks, edits, and revises for correct information, appropriate 
emphasis, form, grammar, spelling, and punctuation. 

F3 Arithmetic - Performs basic computations; uses basic numerical concepts such as 
whole mmibers and percentages in practical situations; makes reasonable 
estimates of arithmetic results without a calculator; and mes tables, graphs, 
diagrams, and charts to obtain or convey quantitative information. 

F4 Mathematics - Approaches practical problems by choosing appropriately from a 
variety of mathematical techniques; uses quantitative data to construct logical 
explanations for real world situations; expresses mathematical ideas and concepts 
oralty and in writing; and understands the role of chance in the occurrence and 
prediction of events. (See footnote on page 2-3.) 

F5 Listening - Receives, attends to, interprets, and responds to verbal messages and 
other cues such as bo^fy language in ways that are appropriate to the purpose; for 
example, to comprehend, to learn, to critically evaluate, to appreciate, or to 
support the speaker. 

F6 Speaking - Organizes ideas and communicates oral messages appropriate to 
listeners and situations; participates in conversation^ discussion, and group 
presentations; selects an appropriate medium for conveying a message; uses verbal 
language and other cues such as body language appropriate in style, tone, and 
level of complexity to the audience and the occasion; speaks clearly and 
conununicates a message; understands and responds to listener feedback; and asks 
questions when needed. 



2-8 



Hunking SkiUs 



F7 Creative Thinking - Uses imagination freely, combines ideas or information in new 
ways, makes connections between seemingly unrelated ideas, and reshapes goals in 
ways that reveal new possibilities. 

pg Decision Making - Specifies goals and constraints, generates ahematives, considers 
risks, and evaluates and chooses best alternative. 

F9 Problem Solving - Recognizes that a problem exists (i.e., there is a discrepancy 
between what is and what should or could be); identifies possible reasons for the 
discrepancy, devises and implements a plan of action to resolve it; evaluates and 
monitors progress; and revises plan as indicated by findings. 

FIO Seeing Things in the Mind's Eve - Organizes and processes symbols, pictures, 
graphs, objects or other information; for example, sees a building from a 
blueprint, a system's operation fi-om schematics, the flow of work activities from 
narrative descriptions, or the taste of food from reading a recipe. 

Fll Knowing How To Learn - Recognizes and can use learning techniques to apply 

and adapt new knowledge and skills in both familiar and changing situations and is 
aware of learning tools such as personal learning styles (visual, aural, etc.), formal 
learning strategies (note taking or clustering items that share some characteristics), 
and informal learning strategies (awareness of unidentified false assumptions that 
may lead to faulty conclusions). (See footnote on page 2-3.) 

F12 Reasoning - Discovers a rule or principle underlying the relationship between two 
or more objects and applies it in solving a problem; uses logic to draw conclusions 
from available information; extracts rules or principles from a set of objects or 
written text; applies rules and principles to a new situation or determines which 
conclusions are correct when given a set of facts and a set of conclusions. (See 
footnote on page 2-3.) 

Personal Qualities 

F13 Responsibility - Exerts a high level of effort and perseverance tpward goal 
attainment; works hard to become excellent at doing tasks by setting high 
standards, paying attention to details, working well and displaying a high level of 
concentration even when assigned an unpleasant task; and displays high standards 
of attendance, punctuality, enthusiasm, vitality, and optimism in approaching and 
completing tasks. 

F14 Self-Esteem - Believes in own self-worth and maintains a positive view of self; 
demonstrates knowledge of own skills and abilities; is aware of impact on others; 




and knows own emotional capacity and needs and how to address them. (See 
footnote on page 2-30 



F15 Social - Demonstrates understanding, friendliness, adaptability, empathy and 
politeness in new and on-going group settings; asserts self in familiar and 
unfamiliar social situations; relates well to others; responds appropriately as the 
situation requires; and takes an interest in what others say and do, 

F16 Sel^Manapement - Assesses own knowledge, skills, and abilities accurately; sets 

well-defined and realistic personal goals; monitors progress toward goal attainment 
and motivates self through goal achievement; exhibits self-control and responds to 
feedback unemotionally and non-defensively; and is a "self-starten" 

F17 Integritv/Honestv - Can be trusted; recognizes when faced with making a decision 
or exhibiting behavior that may break with commonly-held personal or societal 
values; understands the impact of violating these beliefs and codes on an 
organization, self; and others; and chooses an ethical course of action, (See 
footnote on page 2-3,) 



2-10 

ERIC 



APPENDIX D: 
Prototype Data Collection Instrument 
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APPENDIX E: 
Occupations Worksheet 



OCCUPATION = title as approved by State Board of Education 

CAT = categorized by amount of training/e>q>erience required from least to most 

COMMENTS = notes from Occupational Outlook Handbook 

DEMAND = number of openings expected annually as shown in appendix to State Plan 
FROM 2v = "supply" expected from secondary vocational and applied technology education 
SUPPLY = total projected supply 

%2vt = proportion of supply expected from secondary vocational and applied technology 
SOCAT = name of student occupational competency achievement test that is available 
socat$ = projected cost per student to administer SOCAT 
COST = FROM 2v x socat$ (i.e., cost to adminster SOCAT if all 2v take the test) 
SCANS = shows whether or not SCANS developed list of tasks and skills for the job 
CERTIF = agency that provides non-education based assessment and/or skill certification 
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APPENDIX F: 

Tables Matching SOCATs to Essential Elements for Specific Courses 

BILLING CLERK 
CARPENTER'S HELPER 
CHILD CARE WORKER 
COMPUTER OPERATOR 
DENTAL ASSISTANT 
DIESEL MECHANIC 
DRAFTER 

ELECTRICIAN'S HELPER 
MACHINIST 

MAINTENANCE REPAIRER: Small Engine 

MEDICAL ASSISTANT 

OTHER CONSTRUCTION HELPER 

OTHER HEALTH WORKER: Home Health Aide 

OTHER SECRETARIES 

PLUMBER'S HELPER 

PRODUCTION CLERK X 
WELDERS & CUTTERS 
WORD PROCESSING TYPIST 
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Business Data Processing 



Written Test - 3 hrs 168 items 



1 general software applications 



2 systems/computer programming 



3 computer literacy 



4 computer operations 



5 business related computer functions 



Performance Test - 325 $1.00 



1 backup diskette 



2 use of word processing,spreadsheeU. 



& database packages 



E= Business Computer Applications I 



BILLING CLERKS 



Specific courses & eei 



El 



E1&4 



El 



Elc 



E5&6 



E2 
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Caipcntrv - 533 hrs 




Written Test - 3 hrs 153 items 


Specific courses & eei 


1 blueprint reading 




2 preparation of specifications 


rr4 


3 building materials 


bblb 


4 hand & power tools 


BBoA 


5 foimdations 


EEIL&U 


6 forms 


EblC 


7 rough framing 


bblb, rr/ 


8 roof components 


bblr 


9 interior & exterior finish 


bblH 


10 stair construction 


Ebll 






Performance Test - 233 $1025 




1 rafters 


Eblb, rr / 


2 window frame construction 


bblb, rr/ 


3 layout 


rr4 


4 tool & material identification 


bblti, rr^4 


5 use of transit 




6 blueprint reading 


bblA. rrl 


7 stairway stringer 


17177 

rr / 


8 sole plate 












EE= Building Trades 




FF« Construction Carpentry 




BB« Introduction to Construction Careers 
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CMd Care Services 



CHILD CARE WORKER 



Written Test - 3 hrs - 175 items 



Specific courses & eci 



1 young child development 



2 relationships with families 



FIA 



F2D 



3 learning environment 



F3A-D 



4 advanced social/emotional development 



FIB &C 



5 intellectual competence 



6 professionalism 



7 guidance 



8 program management 



Performance test - 3 hours - $0.00 



FIB 



2&5 



FlC 



F4F 



1 development of learning activity 



F4C 



2 development of lesson plan 



F4B&E 



3 employment application 



4 self-evaluation 



F4A 



F=Child Care and Guidance, Management and Services 



Unlettered =^ general requirements for all courses in Home Economics 



00 



Computer and Information Sciences 


COMPUTEK UrfcKAlUK 


Written Test - 


Specific courses & eel 


information technology uses 


C4A-D 


related math 


U3A 


computer programming language « 


UoB&D 


applications software 


E5 


computer literacy 


OLA, 03i> 


computer user skills 




computer programming concepts 








Performance Test - 4 hrs $2.00 




spreadsheet application 


E3 


word processing/text writer application 


EID 


coding in BASIC, FORTRAN, or PASCAL 


G3 


database application 


ULA, t4A 


diskene backup 












C= Business Information Processing 




G» Business Computer Programming I 




E= Business Computer Applications I 






Dcntai Assisting 


DENTAL AJS3IO IAN lo 


Written Test - 3 hrs - 166 items 


Specific courses & eei 


1 dental materials 


t?0 11 10 

r 5,1 14^ 


2 laboratory skills 


/^i ft 


3 cbairside skills 


rl 


4 dental sciences/related skills 


F13,2,3 


5 dental radiology 


F4 


6 related theoiy & skills 








Performance tests - 1.75 hrs $2.00 




1 instruments 


I?Q 1 1 

ro,ll 


2 radiographs 


r4 


3 handwashing 


Fll 


4 autoclave 




5 insurance forms 


t?1 A 

rlU 


6 syringes 












F= Dcntai Assistant 




G= Dental Laboratoiy Aide 
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Diesel Engine Mechanics - 5^ hrs 




Wntten Test - 3 hrs 179 items 


opeCUlC LOUTbCb OL CCI 


1 servicing diesel fuel system 


\^v*tUj nno 


2 rebuilding engines 


inri*t 


3 shop equipment 




4 engine disassembly 




5 maintaining & servicing cooling & lube systems 


i^c*fUj HUD 


6 servicing intake & exhaust systems 


HH6 


7 testing & servicing electrical system 


HH7&8 


8 maintenance testing & engine operation 


HHl 


10 maintaining & servicing cylinder heads 


11 servicing flywheel/clutch/transmission 


cc3D 


12 safety ^ 


24 






Performance Test " 2^ hrs $5.00 




1 batteiy testing 


CC3D 


2 clutch adjustment 




3 drilling 


tl&5 


4 tapping 


T1&8 


5 engine repair 


HH4 










CC= Introduction to Transportation Service Careers 


HH« Diesel Mechanics 




T«Machine Shop 




Unlettered* general requirements for all courses in Trade & Industry 




Drafting 


no A tTTPP^ 

UKAr ii;::.Ko 


Written Test - 3 hrs - 120 items 


opecuic courses oc. cci 


1 interpretation of drawings 


AjA, AA4 


2 machine drawing 


A A O 


3 architectural drawing 


A A 

AAo 


4 mathematical calculations 




5 electronic drawing 


A 


6 sheet metal drawing 


AAO 


7 mapping & cartography 


A A11 

AALl 


8 computer-aided drawing 


VIO A A7 
A3L, AA/ 






Performance Test - 3 hrs $0.00 




1 production/detail 


A A 1 1 ^ 

AAi-ia 


2 specialty areas of: 




electrical, piping, architectural 


A AiC 

AAo 


3 orthographic projection 




4 auxiliaiy 


A A 

AAj 


5 threads 












X« Introduction to Graphic Communications Careers 


AA« Drafting 





y4 



Electrical Occupations 


ELECTRICIAN'S HELPERS 


Written Test - 3 hrs - 183 items 


Specific courses & eei 


1 motors & motor controls 


L2A&C 


2 National Electrical Code 


HlC 


3 materials 


7D 


4 tools 


7D 


5 blueprint reading & estimating 


HID 


6 alternating current 


08 


7 direct current 


02 


8 safety 


4 






Performance test - 3 hrs $2750 




1 drawings/diagrams 


HID 


2 wiring 


HIFG, 2A 


3 use of tools 


7E 


4 component mounting 


HIF 


5 economy of time 


2J 






H« Electrical Trades 




L= Introduction to Electrical/Electronics Careers 


0« Industrial Electronics 




Unlettered = general requirements for all courses in Trade & Industrial 



1 Machine Trades - 175 items 83 hrs $12 


MACHINISTS 


Wrinen Test - 3 hrs 


Specific courses & eei 


1 lathes 


T7,U1A 


2 milling 


SA, Tl, UlA 


3 grinding 


SA,T11 


4 benchwork 


UIH 


5 drilling 


SA, Tl, UlA 


6 sawing 




7 related theory 




8 advanced processes (CNC,EDM) 








Performance Test - 53 hrs 




1 lathe operations 


T7, T8, UIF 


2 drilling operations 


T4 


3 milling machine operations 


UlA, UIE&G 


4 measurement 


SB,T2 


5 safety 


4 


6 layout 


T3 


7 benchwork 


UIH 










S= Introduction to Precision Metal Manufacturing Careers 


T« Machine Shop 




U« Metal Trades 




Unlettered = genera! requirements for all courses in Trade & Industry 



S6 



• 



Small Engine Repair 


MAINTENANCE REPAIRER-SMALL | 
ENGINE 


Written Test - 176 items 


Specific courses & eei | 


1 thcoiy and shop arithmetic 


— 


2 ignition 


I5D&6D 


3 compression and lubrication 


I5D&yD, M5, S3 


4 valves, ports, and exhaust 


I5D 


5 engine block components and cooling 


I5D&6D, SI, M3 


6 shop procedures and safety 


4 


7 governors 




8 starters 


I5D,S3 


9 powered equipment mechanisms 


SI 






Performance test - 3 hours - $6.45 




1 engine check 


I6E, M4, S6 


2 carburetor service 


S3 


3 valve service 


S6 


4 ignition system service 


I6E, S3 


S parts mspection, measurement, testing 




6 invoicing parts 


M2 


7 use of tools and equipment 


7D&E 


8 use of time 


2H4,&J 






M= Marine Engine Repair 




S« Small Engine Repair 




I « Introduction to Transportation Service Careers 


Unlettered* general requirements for all courses in Trade & Industiy 
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Medical Assisting • 168 items 5-67 hrs 


MEDICAL ASSISTANTS 


Written Test • 3 hrs 


Specific courses & eei 


1 clinical assisting 


L3 


2 laboratoiy procedures & equipment 


K 


3 medical terminology 


B2C 


4 anatomy & physiology 


BIG 


5 patient instruction 




6 radiology 


N 


7 diagnostic procedures & equipment 


L8, K5 


8 electrocardiography 


I 


9 medical ethics & law 


BIC 






Performance Test - 2.67 hrs 




1 laboratory procedures 


K 


2 diagnostic procedures 


L8, K5 


3 clinical procedures 


L3 


4 general office procedures 


B2 










B^Health Care Science 




1 * ciccirocaTuiograpn lecn 




K= Medic. « Lab Aide 




N« Nursing Assistant, Radiology 




L« Medical Office Assistant 





S8 



Building Construction Occupations - 5S hrs 


OTHER CONSTRUCTION HELPERS 


Written Test - 3 hrs 172 items 


Specific courses & eei 


1 general building construction 


B2D, El 


2 blueprint reading 


B2A, ElA 


3 electrical 


B4, E3 


4 painting & decorating 


B5, E4 


S plumbing & heating 


B6m E5, KIK 


6 general safety 


4 






Performance Test • 3^ $10.00 




1 plumbing 


B6, E5 


2 carpentry 


B3, El 


3 electrical 


B4, E3 


4 masoniy 


B2E&F, E2 










B= Introduction to Construction Careers 




E= Building Trades 




Piping Trades/Plumbing 




Unlettered = general requirements for all courses in Trade & Industiy 



SB 



Home Health Aide - 138 items 5 hrs 


OTHER HEALTH WORKER: HOME 
HEALTH AIDE 


Written Test 


Specific courses & eei 


1 reporting 


Mil 


2 recording 


J7 


3 abbreviations 




4 vocabulaiy 


B2C, C2B 


S first aid 


BIB, CIG, Dll 


6 vital signs 


B4C, CIH, DU 


7 nutrition 


H6 


8 elimination 




9 basic emergency measures 




10 patient care 


Jl-3 


11 positioning 




12 employability sldils 


C2A 


13 safety 


BIB, CIC, Die 






Performance Test 




1 vital signs 


B2C, CIH, DU 


2 infant care 


DD2, E2 


3 bed making 


J6 


4 first aid 


BIB, CIG, Dli 


5 patient transfer 


J3 


6 measurement of fluids 


J7 






M= Nursing Assistant 




B« Health Care Science 




C» Health Occupations Education I 




D>eHealtn uccupations Education ii 




E-Child Dcvclq)ment 




H-Diet Cleric 




J-Home Health Aide 





erJc 
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General Secretarial 


OTHER SECRETARIES 


Wntten Test - 3 hrs - 175 items 


Specific courses & eei 


1 ofiice procedures 




2 filing 


TO rM TJf 

12, DIM 


3 communication skills 


15, B4 


4 ke)4x>arding skills 


BIB&C, Jl 


5 word processing 


E1&2, J2 


6 computational skills 


Eo 






Perionnance test - 3 nrs $1.(X) 




1 transcription 


CID 


2 document processing 


CIC 


3 prioritizing 




4 kej'boarding 


BIB&L, Jl 


6 filing 


12, UIH 










B« Office Administration 




C== Business Information Processing 




D» Business Office Services 




E-Business Computer Applications I 




I -Office Suppon Systems 




J « Microcomputer Applications 
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Plumbing - 6 hrs 


PLUMBER'S HELPER 


Wrinen Test - 3 hrs 156 items 


Specific courses & eei 


1 assembly & layout 


B6A-C 


2 installation 


B6A-C, E5 


3 planning 


K2B 


4 inspecting 


K2E 


5 evaluating 


K2E 


6 maintenance & repair 


B6A-C, K2B, K2E 






Performance Test - 3 hrs $11.70 




1 drawings 


K2A 


2 layout 


E5 


3 rough installation 


B6A-C, K2B 


4 installation of fixture 


E5, K2D 










B« Introduction to Construction Careers 




1^— T^iiilHincT XtoHac 

Duiiuing 1 raucs 




K= Piping Trades/Plumbing 
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Pnxluction Agriculture 


rKUDUmUN CLcKJ^o 


Wntten Test - 3 hrs - 164 items 


Specific courses & eei 


1 fam management & record keeping 


Jl-11 


2 anmial science 


/^1 O 


3 plant science 


Ki-o 


4 leadership & cmployabiiity 


1 oL Z 


5 agricultural mechanics 


r\i 1 1 






Performance test - 2o hrs 55.15 




1 animal science 




2 agricultural mechanics 




3 farm management 


11 2 
Jlo 


4 recordkeeping 


Jo 


5 plant science 


D1 Q 

Kl-o 










J = Agribusiness Management & Mailceting 




Q=Anunal Science 




R=Plant&Soil Science 




D= Introduction to Agricultural Mechanics 




Unlettered = general requirement for all courses in Agricultural Education 
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Welding - 6.13 hrs 


WELDERS & CUTTERS 


Written Test - 3 hrs 151 items 


Specific courses & eei 


1 shielded metal arc welding 


W2 


2 OQrfuel welding 


Wl 


3 welding terms & symbols 


W5, U2H 


4 safety 


4 


5 electricity 


YIA 


6 basic metallurgy 




7 gas metal arc welding 


W3 


8 gas tungsten arc welding 


W3 






Performance Test - 3.13 hrs $5.50 




1 shielded metal arc welding 


W2, U2F 


2 oxyacetylene welding 




3 oxyfiiel cutting 


Wl, U2B 


4 gas metal arc welding 


W3, U2F 


5 gas tungsten arc welding 


W3, U2F 










W=Welding 




U« Metal Trades 




Y« Introduction to Electrical/Electronic Careers 


Unlettered = general requirements for all courses in Trade & Industiy 
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General Office 


WORD PROCESSING TYPIST 


Written Test - 3 hrs - 167 items 


Specific courses & eei 


1 business communications 


15 


2 maQ processing 


14 


3 key boarding 


U 


4 word processing 


E3&6 


5 filing & records management 


12 


6 office procedures 


16 


7 computational skills 


13 


8 mterpersonal & employability skills 


17 






Performance test - 3 hrs $L00 




1 filing & records management 




2 computational skills 


I2A 


3 tabulation 


B 


4 business correspondence 


13 


5 forms preparation 


15 


6 envelope preparation 


04A 


7 communication 


15 










E= Business Computer Applications I 




I = Office Suppon Systems 
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First Quarter Report 

Performance Standards for 
Vocational and Applied Technology Education 



Barbara E. Selke-Kem 




Sponsored Cooperatively by 
Texas Education Agency 

and 

The University of Texas at Austin 
Extension Instruction and Materials Center 
Division of Continuing Education 



October 1, 1991 



Proiect Number 22420025 
RFA Number PD-2 




The project reported herein was perforaied through a contract with the 
Texas Education Agency under the provisions of the Carl D. Perkins 
Vocational Education Act (Public Law 98-524). Contractces undertaking 
such projects arc encouraged to express freely their professional judgment 
in the conduct of the project Points of view or opinion stated do not, 
therefore, necessarily represent official Texas Education Agency position or 
policy. 
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Performance Standards for 
Vocational and Applied Technology Education 

Period Covered: From July 1, 1991 to September 30, 1991 
Major Activities and Accomplishments During This Period 

1 . Nine teleconferences were held with members of the Specific Advisory Groups. 

2. It was determined that Richard Bouton — the designated project manager of the 1990- 
91 project. Research and Development of Performance Standards for Selected 
Vocational Education Programs, and the 1991-92 project. Performance Standards for 
Vocational and Applied Technology Education — ^was too ill to continue. A search for 
a new project manager was carried out and an individual for this role identified. 

Significant Findings and Events 

1 . Richard Bouton conducted three interest sessions at the August Vocational 
Administrators and Supervisors Conference to provide a status report on the project, to 
obtain advice on the draft core performance standards, and to solicit volunteers for the 
feasility study to be conducted in the fall. 

2. Richard Bouton planned and conducted nine teleconferences with members of the 
Specific Advisoty Groups to obtain reactions to and recommendations on the draft 
core performance standards developed to date. 

3 . The status of the progress on the 1990-91 and 1991-92 performance standards projects 
was assessed. When it was determined that Richard Bouton could not continue with 
the performance standards project, all work done to date was collected and reviewed 
and, with the advice and consent of the Texas Education Agency's (TEA^s) directors 
of Vocational Programs and Vocational Special Needs, an alternate plan of action was 
developed. 

4. A search was conducted for a new project manager. A qualified individual with the 
qualifications and ability to carry forward with the project in a timely and quality 
fashion was identified* 

Problems 

1 . Significant delays were created by Richard Bouton's illness and his resulting inability 
to complete the work he had begun in die 1990-91 project and to carry out the 1991-92 
project. 

2. The problems created by Mr. Bouton's departure were compounded by TEA's 
decision to move from June to January the timeline for delivery of the feasibility-tested 
core performance standards. 

Data Collection 

1 . Recommendations on the draft core performance standards were solicited from the 
nine Specific Advisory Groups and from participants in the August Vocational 
Administrators and Supervisors Conference. 




ERIC 



Progress on Evaluation Plans and Procedures 

1 . Evaluation plans and procedures were not addressed during this quarter. 
Staff Employment and Development 

1 . With the advice and consent of the TEA's directors of Vocational Programs and 
Vocational Special Needs, Richard Bouton was dismissed as project manager and the 
search for a new project manager conducted. 

2 . The individual to be recommended as the new project manager was identified 
Future Activities 

1 . Approval for the new project manager will be obtained from TEA's directors of 
Vocational Programs and Vocational Special Needs, and the new manager will be 
hired in the first weeks of the second quarter. 

2 . The project will continue in accordance with the agreed upon timelines and conditions. 
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SECOND QUARTER REPORT 



Performance Standards for 
Vocational and Applied Technology Education 



Cindy Roberts-Gray 
Resource Network 



Sponsored Cooperatively by 
Texas Education Agency 

and 

The University of Texas at Austin 
Exte/ision Instruction and Materials Center 
Division of Continuing Education 



January, 1992 



Project Number: 22420025 
RFA Number: PD-2 



The project reported herein was performed through a contract with the Texas Education 
Agency under the provisions of the Carl D. Perkins Vocational Education Act (Public Law 
98-524). Contractees undertaking such projects are encouraged to ejq^ress freely their 
professional judgment in the conduct of the project. Points of view or opinion stated do not, 
therefore, necessarily represent official Texas Education Agency position or policy. 
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Performance Standards for 
Vocational and Applied Technology Education 

Period Covered: From October 1 to December 31, 1991 

Major Activities and Accomplishments During This Period 

1. A meeting of the General Advisory Committee was conducted October 21, 1991. 
The committee approved a set of draft standards and a preliminary draft of a 
prototype data collection instrument. 

2. Twelve school districts were recruited to participate in the feasibility assessment: 4 
large districts (Brownsville, Cypress-Fairbanks, El Paso, and Garland), 3 medium size 
districts (Amarillo, Lamar Consolidated, and Weslaco), 4 small districts (Bellville, Big 
Spring, East Central, and Lufkin), and 1 district with fewer than 600 students 
(Hutto). 

3. During November and December, site visits were made to all 12 of the school 
districts recruited to participate in the feasibility assessment. 

4. The final report for project year 1991 was completed and submitted in December, 
1991. 

Significant Findings and Events 

1. Information obtained during interviews with school district personnel and review of 
records in the 12 school districts that participated in the feasibility assessment 
indicated need for combining and revising several of the draft standards. School 
district personnel also requested substantial revision in the data collection instrument. 

2. It appears that nearly all of the data elements that would be needed to measure 
performance relative to the standards are elements that districts already collect for 
entry into the PEIMS. 

3. There is substantial concern within the districts about implementation issues. District 
personnel indicated need, for example, for more guidance from the TEA about 
definition of "a coherent sequence of courses" in vocational education programs and 
about programs or procedures that would be needed to pull the required data from 
the PEIMS. 

Problems 

1. The feasibility assessment was originally planned as a two-part study with background 
and baseline data to be collected in the winter of 1991 and a pilot implementation 
of the standards and measures to be conducted in the spring of 1992. Because of 
tea's decision to move the timeline for delivery of the feasibility-tested core 
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performance standards from June to January, both phases of the assessment were 
collapsed into single site visits made in November and December 1991. The core 
standards, measures of performance, and data gathering instrument are based, 
therefore, on guidance received from background research, the ejq>ertise of the 
General Advisory Committee, and critique and suggestions of school district 
personnel. 

The work plan for this project specifies that, in addition to the core performance 
standards, "course specific" standards will be developed as appropriate. Neither the 
project records nor information obtained during the feasibility assessment provided 
clear demand or guidelines for the development of course specific standards. 



Data Collection 



1. Recommendations on the draft core performance standards and the prototype data 
collection instrument were obtained from participants at the meeting of the General 
Advisory Committee in October, 1991. 

2. Interviews, records reviews, and pilot implementation of the data collection 
instrument were conducted on-site in the sample of 12 school districts in order to 
assess the feasibility of the core standards, measures of performance, and data 
collection instrument. 




Progress on Evaluation Plans and Procedures 



The evaluation plan for this project calls for regular and frequent meetings with the 
TEA project director, the General Advisory Committee, and in-house evaluation to 
ensure progress in accord with project plans and timelines. During the period 
reported here, there was one meeting with the General Advisory Committee, two 
meetings with program specialists at UTEIMC, and three meetings with the project 
director, and one meeting with program directors at the TEA. Results of these 
meetings were used to make appropriate adjustments in project procedures. 

Staff Employment and Development 

Cindy Roberts-Gray, Ph.D., of the Resource Network of Austin, began work as 
project manager in October 1991. Her assistant on this project is Sandra Wells. 



Future Activities 



1. Results of the feasibility assessment will guide development of core standards, 
measures of performance, and data collection instrument that will be recommended 
to the TEA and presented at the mid-winter administrators conference in January. 




2. The project will continue in accordance with the agreed upon timelines and 
conditions. 
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Third Quarter Report 

Performance Standards for 
Vocational and Applied Technology Education 



Cindy Roberts-Gray 
Resource Network 




Sponsored Cooperatively by 
Texas Education Agency 

and 

The University of Texas at Austin 
Extension Instruction and Materials Center 
Division of Continuing Education 



April 30, 1992 



Project Number 224200425 
RFA Number PD.2 




The project reported herein was performed through a contract with the 
Texas Education Agency under the provisions of the Carl D. Perkins 
Vocational Education Act (Public Law 98-524). Contractccs undertaking 
such projects are encouraged to express freely their professional judgment 
in the conduct of the project. Points of view or opinion stated do not, 
therefore, necessarily represent official Texas Education Agency position 
or policy. 
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Performance Standards for 
Va:ational and Applied Technology Education 



Period Covered: From January 1, 1992, to March 31, 1992 
Major Activities and Accomplishments 

1. Data collected in the sample of the 12 school districts that participated in the 
feasibility study in November and December 1991 were analyzed and used as a guide 
in revising the draft of core standards, measures of performance, and data collection 
instrument for the evaluation system. 

2. A working paper entitied, "Findings and Recommendations from die Feasibility 
Study," was prepared and submitted to die senior diiector and program directors for 
vocational and applied technology education at the Texas Education Agency on 
January 22, 1992. This report included the revised drafts of core standards, measures 
of perfomiance, and data collection instrument Subsequent to a January 27 meeting 
with the program directors and die director of vocational funding, monitoring and 
compliance, additional revisions were made to ensuie that data are captured to 
measure program access and program success for targeted groups to achieve equity in 
vocational and applied technology education. 

3. On January 30, 1992, at die Mid-Winter Vocational Supervisors, Administrators, and 
Counselors' Conference, presentations were made to describe progress of the project 
and receive input from conference participants to guide implementation planning for 
die evaluation system. 

4. On January 31, 1992, a report entitied, "Performance Standards for Vocational and 
Applied Teclmology Education: Core Standards, Measures of Performance, and Data 
Gadiering instruments," was subnoitted to die project director and die senior director 
for vocational and applied technology education at TEA to distribute to members of 
the Committee of Practitioners for their consideration. 

Significant Findings and Events 

1 . Project staff met widi die senior director of vocational and applied technology 

education and the project director at TEA on February 10, 1992, to review die project 
work plan. At that meeting, it was determined tiiat, given guidance to date from 
vocational programs directors at TEA and results of field visits in November and 
December 1991, it would be more appropriate to use the remaining months of the 
project to address issues related to implementing the core standards and measures of 
performance than to address the issue of course-specific standards. 

Following this decision, additional information regarding implementation issues was 
obtained in meetings on February 12, Febraary 19, and March 20 widi TEA staff who 
have special expertise regarding the state's Public Education Information 
Management System (PEIMS) and die Academic Excellence Indicator System 
(AEIS). 
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2 On February 27, 1992, the Committee of Practitioners approved, with several 

modifications, the eight core standards and measures of performance recommended in 
the report dated January 31, 1992. 

3. In mid-March, 1992, it was learned that the Commissioner of Education judged that 
the initial proposed standards and measures did not adequately address quality 
expectations relating to such items as academic gains, following students from 
enrollment in a coherent sequence of vocational and applied technology education to 
completion, and the enrollment and progress of special populations students in high 
level or rigorous vocational programs. TEA staff selected only three of the eight 
proposed standards and measures to be presented to the State Board of Education for 
its consideration. 

Problems 

1. At the close of this third quarter of the project, it is not clear what direction should be 
taken in the remaining months to produce information and products that will be as 
useful as possible in implementing an evaluation system tiiat will meet tiie 
requirements of tiie Perkins Act and adequately address quality expectations for 
vocational and applied technology programs. 

Data Collection 

1. Recommendations on the draft core performance standards and the prototype data 
collection instrument were obtained from directors of vocational and applied 
technology education programs at the TEA in January 1992; from participants at the 
Mid-Winter Vocational Supervisors, Administrators, and Coiinselors' Conference in 
January 1992; and ftom TEA staff famiUar with PEIMS and the AEIS in February 
and March 1992. 

Progress on Evaluation Plans and Procedures 

1 . The evaluation plan for this project calls for regular and frequent meetings with the 
TEA project director, the General Advisory Committee, and in-housc evaluation to 
ensure progress in accord with project plans and time lines. During this quaner, there 
was one such meeting with vocational program directors and the director of 
vocational funding, monitoring, and compliance at TEA (January 27,1992); three 
meetings with the project director (January 27, February 10, and March 1 1, 1992); 
and three meetings with the senior director of vocational and applied technology 
education (February 10, March 11, and March 20). Results of these meetings were 
used to make appropriate adjustments in project procedures. 

Staff Employment and Development 

1 Cindy Roberts-Gray, Ph.D., of the Resource Network of Austin, began work as 
project manager in October 1991. Her assistant on tiiis project is Sandra WeUs. 



115 



Future Activities 



1. Pending guidance from the project director and die senior director for vocational and 
applied technology education at TEA, information obtained during meetings with 
staff at the TEA regarding AEIS will guide further work on this project 

2. It is expected diet the project will continue in accordance witii objectives and time 
lines specified in the approved work plan for this project. 
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